Academic Compass
Learning Environments Survey:
Virtual Learning Environments Report
October 2014
Prepared the by Simon Welsh, Senior Learning Analytics Officer, Learning Technologies, Division of
Student Learning
1
Contents
Executive Summary .................................................................................... 3
Background................................................................................................. 4
1. Respondent Profile ............................................................................... 5
2. Staff Attitudes Toward Technologies..................................................... 8
3. Use and Perceptions of CSU’s Educational Technologies ..................... 13
4. New and Emerging Technologies......................................................... 24
5. Improving the CSU Virtual Learning Environment ............................... 27
6. Conclusions ......................................................................................... 29
2
Executive Summary

Between Monday 29 September and Monday 13 October 2014 inclusive, CSU’s academic
staff were invited to complete a survey on the University’s learning environments – both
physical and virtual. This report focuses on academics’ attitudes, usage, perceptions and
expectation of the Virtual Learning Environment

178 CSU staff completed the survey, with around 80% being in direct teaching roles.
Responses were received from across the institution, but the Faculty of Arts is significantly
under-represented in these findings

We may be observing a shift in attitudes among staff to being more embracing of new
educational technologies. However, despite increasingly positive dispositions toward
educational technology, we also observe a shift in the perceived benefits of such
technologies. In the 2010 Staff Educational Technology Survey, the primary benefit of
educational technologies was most commonly seen as providing a means to improve the
quality of teaching. In this current survey, academics are most likely to see the main benefit
as being a means of improving student accessibility and convenience. Thus, on the one hand
we have a strong disposition toward technology, but on the other we have a weakening of
the view that it supports improved teaching

To redress this loss of faith in educational technologies as a means to enhance teaching, we
can better connect, in a holistic way, with the needs, expectations and suggested
improvements expressed by academics in this survey. This means:
i.
improving and sustaining availability or reliability;
ii.
enhancing the user-friendliness and intuitiveness of technologies offered;
iii.
improving and expanding the pedagogical affordances and ability to support quality
learning & teaching; and
iv.
enhancing the professional development and technical support around these
systems, as this a critical part of the experience for academics. The survey findings
suggest there is a need and desire among academic staff for “deeper” conversations
around how educational technologies can be used to enhance learning and teaching,
and what constitutes good online learning and teaching. This pertains to all teaching
staff, including Sessional staff where there may be a particular opportunity to
achieve better engagement with and facilitate the uptake of online learning

Figure ES-1 provides a summary of the key factors that appear to drive academics’
perceptions of the value or worth of educational technologies
3
Background
Academic Compass is a series of surveys which will regularly explore the needs, expectations and
perceptions of CSU’s academic staff. The Compass aims to provide a mapping of current issues
related to academic practice, in particular learning and teaching, and point the way forward as CSU
charts new directions.
This edition of the Compass addressed academics’ perceptions and expectations of CSU’s learning
environments – both physical and virtual. The survey sough feedback on:

the capacity of our learning and teaching spaces and the online learning environment to
support teaching and quality learning for our students; and

how our learning environments need to evolve, so that we can plan and provide direction
for the future.
This report contains in-depth findings from the section of the Learning Environments Survey relating
to Virtual Learning Environments (VLEs), including:
1. Respondent profile
2. Attitudes toward technologies
3. Use and perceptions of CSU’s educational technologies
4. New and emerging technologies
5. Improving the CSU VLE
6. Conclusions
A copy of the online survey (administered via Survey Monkey) can be viewed here: Learning
Environments Survey
Comparisons with the 2010 Staff Educational Technology Survey are made where relevant and highlighted.
4
1. Respondent Profile
Between Monday 29 September and Monday 13 October 2014 inclusive, 178 CSU staff members
participated in the Academic Compass survey on learning environments.
The profile of respondents is outlined in the following figures, key points include:

Figures 1.1 to 1.3 show the respondent profile as a percentage of staff who completed the
respective question. In all cases, profile questions were only completed by 150 of the total
178 staff who commenced the survey

The majority of survey participants who provided profile information were Lecturers or
Senior Lecturers (66% of sample combined). A further 15% were casual/sessional academics
or Associate Lecturers. This means that over 80% of the sample was comprised of staff with
direct teaching roles. (See Figure 1.1)

Around a quarter of staff did not record their primary Faculty (choosing either the “prefer
not to say” option or skipping the question)

Based on estimation of academic staff “head count” in the Faculties (using Office of Planning
and Audit data), we see a significant under-representation in the survey of staff from the
Faculty of Arts among those who provided profile information. The Faculty of Arts
comprised only approx 15% of the survey sample while contributing over 30% of the CSU
academic staff head count. The Faculty of Business is also somewhat under-represented
(around 11% of survey compared with over 15% of total head count). Conversely, the
Faculty of Science is over-represented (around 39% of sample compared with less than 35%
of total head count). (See Figure 1.2)

Among those who did complete the profile information, male staff are somewhat underrepresented in the survey sample compared with total CSU academic head counts (using
Office of Planning and Audit data), comprising around 39% of the sample compared with
approximately 44% of head count. (See Figure 1.3)
Throughout this report cross-tabulations are provided of responses by academics’ current
role/position. For these analyses only the five most prevalent role categories are included.
However, even within this “top five” substantial differences in group size should be noted:
 Sessional staff (13% of staff who completed the survey)
 Lecturer (45%)
 Senior Lecturer (21%)
 Associate Professor (6%)
 Leadership/admin staff (4%)
5
Figure 1.1 – Respondent Profile by Current Position (n=150)
Figure 1.2 – Respondent Profile by Faculty (n=150)
6
Figure 1.3 – Respondent Profile by Gender (n=150)
7
2. Staff Attitudes Toward Technologies

With regard to academics’ dispositions toward technology generally, we find:
o just over 60% of staff could be described as “early technology adopters” (ie love new
technologies and/or use them before most people). This makes an interesting
comparison to the 2010 Educational Technology Survey where only approx 40% of
staff identified as early adopters. This may suggest a shift in attitudes to be more
embracing of educational technology (see Figure 2.1);
o there are differences in dispositions to technology by academics’ role. Here we see
that casual/sessional staff, Senior Lecturers and leadership/admin staff are all
reasonably consistent in their dispositions to technology with around 67%-74% of
each being described as early adopters. Lecturers are somewhat less embracing of
technology with approximately 55% classified as early adopters, however, the
comparatively large size of this group compared with other role categories means
that this result is best compared with the overall average. Thus, we do see that
Lecturers, as a group, are slightly below average in terms of positive dispositions to
technology compared with the academics generally (55% early adopters for
Lecturers vs 61% for overall). Associate Professors show the least interest in new
technology. No Associate Professors were in the top group for propensity to
embrace technology and only one third would fall into the early adopters
classification (see Figure 2.2); and
o there some differences by Faculty with the Faculty of Arts being less positively
predisposed to technology (around 45% early adopters) and the Faculty of Business
being the most positively predisposed (nearly 70% early adopters) (see Figure 2.3)

Staff were asked what they saw as the single most important benefit of using educational
technology. We found:
o staff most commonly (44%) see educational technologies as a convenience tool for
students first and foremost (i.e. enhancing access to materials and resources). Only
a quarter of survey respondents see the primary benefit of educational technology
as being a key means of enhancing the quality of learning and teaching. This
suggests that work is required to raise the awareness and understanding of the
pedagogical affordances that many educational technologies offer. That is, there
may be a need for “deeper” conversations around how such technologies can be
used to enhance learning and teaching, as opposed to simply showing staff how to
use the functions (see Figure 2.4). Interestingly, in the 2010 Staff Educational
Technology Survey, “Improving the quality of my teaching” was the most commonly
given benefit of education technology (44%);
8
o
there were differences among the perceived benefits of educational technologies
based on:
 role – where we see a consistent increase in the proportion of staff rating
“Improving the quality of my teaching” as the most important benefit of
educational technologies as we move from casual/session staff (11%) to
Lecturers (21%), Senior Lecturers (38%) and Associate Professors (44%).
“Communication with students and co-teachers” was relatively important
for sessional and leadership/admin staff with around 16% of staff in both
categories rating this as their primary benefit. “Personal management” was
strongly associated with sessional staff (approx 10% rating is as the main
benefit), while “Preparing my students for the workplace” was not
mentioned at all by Senior Lecturers or leadership/admin staff (see Figure
2.5);
 faculty – only 7% of staff from the Faculty of Education rated “Improving the
quality of my teaching” as the most important benefit of educational
technologies. This compares with around 18% in the Faculty of Arts and just
over 30% in both of the Faculties of Business and Science. The Faculty of
Education showed the greatest diversity in the views on the benefits of
educational technology, being the only Faculty where all categories of
potential benefits – other than “No benefits” – were mentioned by staff.
However, even in Education, the benefit to student accessibility was far and
away the most commonly cited. “To better prepare students for the
workplace” was cited by staff from all Faculties except the Faculty of Arts,
where there was a stronger emphasis on “Communication with students and
co-teachers” (around 18%). “Personal management” was only mentioned by
the Faculties of Education and Business (around 6% of staff in both) (see
Figure 2.6); and
 these findings highlight the need to always consider educational technology
developments from multiple perspectives with distinct differences in desired
benefits appearing by Faculty and by the role of the intended user
9
Figure 2.1 – Dispositions Toward Technology
Figure 2.2 – Dispositions Toward Technology by Role
10
Figure 2.3 – Dispositions Toward Technology by Faculty
Figure 2.4 – Perceived Benefits of Educational Technology
11
Figure 2.5 – Perceived Benefits of Educational Technology by Role
Figure 2.6 – Perceived Benefits of Education Technology by Faculty
12
3. Use and Perceptions of CSU’s Educational Technologies

Academics were asked about the frequency with which they currently use a range of
educational technologies and the frequency with which they would like to use those same
technologies. Figure 3.1 shows a comparison of current vs preferred usage based on
average usage ratings for each technology (where 1 = never, 2 = about once per session, 3 =
every few weeks, 4 = weekly and 5 = more than once per week). Here we see that for every
tool included, on average, our academics would like to use that tool more than they
currently do. Areas where there is the largest under-utilisation of technologies (c/f
preferred rates of usage) include Adobe Connect, audio and video clips, chat rooms and CSU
Replay

The extent to which online learning is being utilised in both distance and on-campus
subjects was explored. Here we found that:
o
around 4% of staff indicated that their distance subjects contain no online
components, excluding the subject outline (if staff who responded “not applicable”
are removed from the analysis, the figure is 5% with no online components).
Overall, around 13% of staff (17% if “not applicable” responses are removed) have
no or only mandatory online components in their distance subjects beyond the
subject outline (see Figure 3.2);
o
the Faculties of Education and Business appear to be the most embracing of online
learning in distance subjects. Education has the highest proportion of academics
with all of their distance subjects being delivered totally online (62%). While,
Business is the only Faculty with no academics reporting that they have no online
components in any of their distance subjects (see Figure 3.3);
o
approximately 16% of Sessional staff and 17% of Leadership/admin staff report
having no subjects with any online component (beyond the subject outline). The
former is, perhaps, of more interest as Sessional staff could be expected to carry the
greater teaching burden/volume compared with Leadership/admin staff. This
suggests there may be an opportunity to better engage with and support Sessional
staff to promote and facilitate the uptake of online learning (see Figure 3.4)
o
as may be expected, there is generally less uptake of online learning in relation to
on-campus subjects. Interestingly, however, only around 6% of academics (8% if
“not applicable” responses are removed) indicate that they have no online
component in their on-campus subjects (excl subject outlines). In terms of overall
numbers, this means that of those who responded to the survey only an additional
three academics have no online component in their on-campus subjects (10
academics in total) compared with distance subjects (7 academics in total).
Furthermore, around 11% of academics (15% if “not applicable” responses are
removed) report that all their on-campus subjects are delivered totally online (see
Figure 3.5). This, of course, raised the questions: how have these academics
interpreted “totally online” and to what extent (if any) do these subjects make use
of on-campus facilities to provide a subject experience that students may expect of
an on-campus subject? Unfortunately, these questions cannot be answered from
this survey (see Figure 3.5)
13


o
the Faculties of Education and Business again appear to be the most embracing of
online learning in on-campus subjects – they were the only Faculties not to have any
academics with no on-campus subjects using an online component. Indeed, 24% of
academics from Education report that all of their on-campus subjects are totally
online – ie Education alone accounts for around two-thirds of all academics who
report being totally online in all their on-campus subjects (see Figure 3.6);
o
there may be an increase in online learning in on-campus subjects as move from
those subjects taught by Sessional staff to Lecturers and Senior Lecturers. However,
the differences are not substantial and the inconsistencies in the sizes of these
groups in the survey sample may be a factor in these results (see Figure 3.7)
Academics were asked about their reasons for using Interact. In Figure 3.8, we see that
(behind usage of Interact being made a requirement), the most commonly cited reasons fell
into two categories reflective of the perceived benefits of educational technology more
broadly:
o
student accessibility or convenience – to allow access to resources such as
supplementary resources (67%), lecture notes/handouts (61%) and audio/video
resources (56%); and
o
improving teaching and learning – increase the opportunities for communication
(59%) and increase flexibility in teaching and learning (57%)
Academics’ experiences of Interact were investigated, in terms of functionality, availability
and overall use. It was found that:
o
generally speaking, feedback on Interact is positive – although academics’ responses
are less positive than students’ responses to the same questions in the 2014 Student
Educational Technology Survey. Availability is the most strongly performing factor,
with just over 70% of academics agreeing or strongly agreeing that their experiences
on this factor have been positive. Experiences of functionality and overall use are
relatively similar with 57% agreeing/strongly agreeing functionality has been positive
and 60% agreeing/strongly agreeing that overall use has been positive (see Figure
3.9);
o
differences in experiences of Interact emerged based on gender, Faculty and role:

across all factors, male academics were more likely to be positive in their
assessment of Interact than their female colleagues (see Figure 3.10);

academics in the Faculty of Business were more likely to be negative in their
assessments (across all factors) than their colleagues in other Faculties (see
Figure 3.11);
14

the small number of Leadership/admin staff in the survey were most
positive about Interact across all factors. Interestingly, for Sessional staff, on
average, their overall use assessment of Interact appears to be most closely
linked to their experience of its functionality (ie approx 52% positive on both
factors). For Senior Lecturers, on average, the overall use assessment is
lower than the assessment of both functionality and availability – suggesting
perhaps that another factor or factors in the experience of Interact may be
important for this group (see Figure 3.12). Indeed, in Section 5 on Improving
the Virtual Learning Environment, we do see a third factor beyond
functionality and availability emerge as important, being training and
support;

Academics’ use of InPlace was investigated and it was found that 24% report using InPlace
(see Figure 3.13). When asked how they would like to use InPlace, only 26 academics
responded and of that number 35% (9) indicated they didn’t know enough about it and
would like to know more. Only 7 respondents indicated a desire to expand InPlace beyond
its current purpose to address either communication and collaboration or to support
learning and teaching for students on placements (see Figure 3.14)

Academics were asked would be their main purpose in using technologies with non-CSU
colleagues. Here we see that 80% of academics see sharing information around shared
projects as the primary purpose. This was followed by providing two-way communication
spaces (65%). Each response category, however, was identified as important by over 30% of
staff, suggesting that it is best to consider that there is a range of purposes in interacting
with non-CSU staff that our technologies need to support (see Figure 3.15)
15
Figure 3.1 – Academics’ Current vs Preferred Usage of CSU Educational Technologies
Figure 3.2 – Online Components in Distance Subjects
16
Figure 3.3 – Online Components in Distance Subjects by Faculty
Figure 3.4 – Online Components in Distance Subjects by Role
17
Figure 3.5 – Online Components in On-Campus Subjects
Figure 3.6 – Online Components in On-Campus Subjects by Faculty
18
Figure 3.7 – Online Components in On-Campus Subjects by Role
Figure 3.8 – Reasons for Using Interact
19
Figure 3.9 – Academics’ Experiences of Interact
Figure 3.10 – Academics’ Experiences of Interact by Gender
20
Figure 3.11 – Academics’ Experiences of Interact by Faculty
Figure 3.12 – Academics’ Experiences of Interact by Role
21
Figure 3.13 – Current Usage of InPlace
Figure 3.14 – What Staff Would Most Like to Use InPlace For (n=26, most common open responses
cited)
22
Figure 3.15 – Using Technologies to Collaborate with Non-CSU Colleagues
23
4. New and Emerging Technologies

Figure 4.1 shows the features of Interact2 that staff are interested in and demonstrates that
there is interest in a wide range of capabilities. This would suggest that our challenge now
lays in engaging with staff to explore the pedagogical implications and opportunities of such
capabilities in order to leverage Interact2 as one means of enhancing learning and teaching
at CSU

Staff were asked what sort feedback they would most value from Learning Analytics tools
to support adaptation in their learning design and teaching. In Figure 4.2 we see that just
over half of those who responded would like quantitative insight on when/how often
students are accessing learning resources and tools, and how they using them, in order to
help staff determine which resources/tools are “working” and which aren’t. Just under a
fifth of staff would value qualitative insight on how well students are learning and whether
they are engaged with subject and its resources. The next most common response (15%)
was that staff simply weren’t sure what to expect

Figure 4.3 shows the “top five” most common categories of suggestions made by academics
for the adoption of new educational technologies by CSU. Even the most prevalent of these
categories (let alone the specific tools mentioned), however, was only raised by 12% of
people who actually responded to this question (which equates to less than 5% of the total
sample). Thus, what Figure 4.3 may really be telling us is that over 60% of academics did not
respond to this question, perhaps because they do not hold a clear view on what
technologies we should adopt (indeed, a number articulated this concern in their response
to this question). This suggests the need for in-depth engagement with academics (or
groups thereof) when considering the adoption of specific technologies/tools to ensure
informed feedback can be obtained
24
Figure 4.1 – Interest in Learning and Teaching Capabilities within Interact2
Figure 4.2 – Most Useful Learning Analytics Feedback to Support Adaptation in Learning Design
and Teaching (n=85, most common open responses cited)
25
Figure 4.3 – Top 5 Technologies Staff Would Like to See Adopted at CSU (n=66)
Category
Responses
Tools to enhance communication and collaborative learning between
students, eg (but not limited to): UCroo, PRAZE, Peerwise, Second Life,
8
improved forums
Improved e-portfolio, eg (but not limited to): ThinkSpace, PebblePad
5
Plus
Tools to support video conferencing with students
4
Tools that enable better/easier use of video learning resources
3
Adaptive learning tools (eg clicker systems, adaptive release)
3
26
5. Improving the CSU Virtual Learning Environment

Staff were asked what they see as the key improvements for our online learning
environment. In Figure 5.1 we find that two common themes emerged:
o that improvements to the online learning environment focus on:
 providing students with learning resources that take better advantage of the
affordances of online delivery, as well as ways of connecting students (in
particular, improved forums) and;
 enhancing the “user-friendliness” of systems for students and staff to make
systems more intuitive to use; and
 the availability/reliability of systems
o that a major improvement lay not in the systems themselves, but in the training,
professional development and support that surrounds them – this perhaps also
relates to the earlier discussion around the need for deeper conversations on the
nexus of technology and learning/teaching; and
o the MSI system and associated processes also raised criticism

Continuing this theme of development and support, Figure 5.2 represents the key responses
to the question regarding perceived barriers to moving subjects online. Here we find
themes around a) what it actually means to “move a subject online” (whole or in-part?
What are the expectations?) and b) what it means to provide quality teaching online and to
support quality student learning. Further key issues were raised around:
o having the time to make adjustments to subjects to make them suitable for online;
and
o ensuring that technologies are accessible/reliable for students with varying qualities
of internet connectivity, and that students are engaged in and kept informed about
what’s happening
Figure 5.1 – Improving the Online Learning Environment (n=92)
Ability Academics Adobe Connect
Faster
Forum Functionality I mproved
I ntegration I nteract Level
M aterials
Formatting
M anagement
O n-line Professional Development Recording
Sessions
StudentsSubject O utline
Support Technologies Tools
Speed
Training Trying
Upload
User Friendly Video
27
Figure 5.2 – Barriers to Moving Subjects Online (n=84)
28
6. Conclusions
The results of this Academic Compass survey suggest that we may be observing a shift in attitudes
among staff to being more embracing of new educational technologies. We now find that 60% of
academics who completed the survey could be classified as “early technology adopters”, compared
with 40% who met such a classification in the 2010 Staff Educational Technology Survey. We also see
a preference among academics to use a range of educational technologies more frequently than
they currently do.
Despite this increasingly positive disposition toward educational technology, we have observed a
shift in its perceived benefits – going from being widely seen as primarily a means to improve the
quality of teaching to a means of improving student accessibility and convenience first and foremost.
Thus, on the one hand we have a strong disposition toward technology, but on the other we have a
weakening of the view that it supports improved teaching.
This may be a function of two factors:
i. increasing expectations of technology; and
ii. dissatisfaction with the educational affordances and/or outcomes of technologies experienced.
That is, educational technologies may not have delivered (for whatever reasons) the opportunities to
improve teaching that were expected and this is leading to a lessening of the “faith” in such
technologies to deliver meaningful new affordances for teaching and to seeing them more as a
convenience tools for students.
In terms of re-building this faith, the needs, expectations and suggested improvements expressed by
academics in this survey are illuminating. Figure 6.1 provides a summary of the key factors that
appear to drive academics’ perceptions of the value or worth of educational technologies.
Figure 6.1 – Drivers of Perceived Value in Educational Technology
Functionality
Availability and
reliability
Dissatisfier
User-friendly and
intuitive
Professional development and support
Pedagogical
affordances and
quality learning &
teaching
Satisfier
29
Figure 6.1 says that there are three critical aspects of the functionality of any educational technology
for academics:
 its availability or reliability – does it work as it should and when it should?
 its user-friendliness and intuitiveness – does it work in a way that is easy to understand,
obvious to engage with and easy to explore?
 its pedagogical affordances and ability to support quality learning & teaching – does it do the
things I want/need it to do as a teacher?
However, these three factors are not equal in their assessment of the worth of a technology – they
exist along a spectrum from being a “dissatisfier” to be a “satisfier”. Availability and reliability is
positioned strongly at the “dissatisfier” end of the spectrum meaning that we can never truly satisfy
through system availability/reliability, we can only dissatisfy. That is, no-one ever goes home from
work saying they had a great day because the LMS worked and stayed on all day. It is expected. We
can only dissatisfy this expectation because anything less than 100% availability is a failure.
Conversely, pedagogical affordances and the ability to support quality learning & teaching can be a
“satisfier” as it addresses a deeper, underlying need – to support the effectiveness and efficiency of
the academic in their job and help them meet their responsibilities and accountabilities. The degree
to which we can deliver this through an educational technology will profoundly influence the
satisfaction with such a tool.
User-friendliness and intuitiveness sits in the middle of the spectrum, meaning that we can satisfy to
a degree through improved design and usability but such improvements are becoming expected.
While these three aspects of functionality provide a high-level guide for the development and
implementation of new educational technologies, just producing highly functional systems that meet
these three criteria is not sufficient. For academics, the professional development and technical
support (especially when things go wrong) around these systems is just as critical and is part of the
experience of the technology.
Indeed, the survey findings suggest there is a need and desire among academic staff for “deeper”
conversations around how educational technologies can be used to enhance learning and teaching,
and what constitutes good online learning and teaching, particularly in relation to a) the move to
place all subjects online and b) the introduction of Interact2 and its expanded capabilities compared
with the current Learning Management System (Sakai). Our challenge is to engage with teaching
staff to explore the pedagogical implications and opportunities of educational technologies in order
to leverage such technologies as one means of enhancing learning and teaching at CSU. That is, the
development and support we provide around educational technologies needs to be deeper than
simple “user guide”-type information (although that is still important) to delve into a) what
constitutes quality online teaching and b) how the available and emerging technologies support and
enable a variety of pedagogies and innovation in teaching.
The need expressed above pertains to all teaching staff, including Sessional staff where there may be
a particular opportunity to achieve better engagement with and facilitate the uptake of online
learning.
30