Ascilite 2010 Report:

This year’s ascilite conference was held in Sydney, at the Novatel Brighton Beach where two papers were presented, one by Colin, and one by myself, and I am sorry that David was not there, but I think finishing his PhD is enough for anyone.

Beer, C. Clark, K., & Jones, D. (2010). Indicators of engagement. In C.H. Steel, M.J. Keppell, P. Gerbic & S. Housego (Eds.), Curriculum, technology & transformation for an unknown future. Proceedings ascilite Sydney 2010 (pp.75-86). http://www.ascilite.org.au/conferences/sydney10/Ascilite conference proceedings 2010/Beer-full.pdf

Clark, K., Beer, C. & Jones, D. (2010). Academic involvement with the LMS: An exploratory study. In C.H. Steel, M.J. Keppell, P. Gerbic & S. Housego (Eds.), Curriculum, technology & transformation for an unknown future. Proceedings ascilite Sydney 2010 (pp.487-496). http://www.ascilite.org.au/conferences/sydney10/Ascilite conference proceedings 2010/Kenclark-full.pdf

Monday 6th December 2010.

The conference was opened with a keynote from Professor Jan Herrington (UTS) on Authentic Learning and emerging technologies, followed by a plenary session on Blackboard.

After the plenary session I listened to some fascinating papers on LMS system analysis and Blended Learning environments. The afternoon session was spent finalizing the first of the papers to be delivered on Tuesday and Wednesday.

Tuesday 7th December 2009.

The morning keynote was by Dr Lev Gonick who spoke of a very high speed interconnectedness in downtown Cleveland, and the difficulties that surround that project, but he also spoke of the possibilities of these developments.

This followed by the invited speaker, Prof Tom Reeves, who spoke on the challenges of online education, and blended learning environments.

Our first paper, presented by Col, was after the lunch session and was on indicators of engagement, which was well received, and originated some useful discussion.

The rest of the day was spent going to other presentations, and we broke early so that we could get ready for the conference dinner.

Wednesday 8th December 2009.

Whilst somewhat dry from the night before, the morning sessions in my breakout group were interesting and the morning was spent getting ready for the second presentation, due after the morning tea break. At this session I spoke on the opportunities for reflective practice that exists in academic analytics.

The conference was interesting, though there is still a lot of work to do with academic analytics, and it was very good to catch up once again with Shane Dawson who has agreed to be a supervisor for my PhD.  And as Shane said, I need to read, “what is your research question”, and basically write, even if it is crap.  I agree with this, it is the art of writing, nit looking for the ‘gems’ that make a successful writer.

Overall, it was a good conference but I was glad to get home

cheers

Ken

Filed under: ascilite, Indicators Project, self-reflection

Academic Involvement with the LMS: an exploratory study

The following is a paper I have written in conjunction with Colin and David as part of the Indicators Project.

Abstract:
There is growing interest in the use of academic analytics however most of the reported work is being done at the level of institutions, and groupings of courses within those institutions. This study is an exploratory case study aimed at analyzing an academics’ involvement with the Learning Management System, the student’s involvement with the LMS, and the links between the LMS, the academic, and the students.

To read more. (Opens as a pdf)

The presentation for ascilite 2010:

Filed under: ascilite, Indicators Project

Moodlemoot 2010 slides

The presentation at Moodlemoot went well. Col presented his paper before mine, and we have a few contacts interested in the Indicator’s Project. I spoke on my own experience as an academic, and the dichotomy between what an academic says he does, and what the stats actually demonstrate what he does. This created some interest in data mining, though the concept in examining one’s own practice was talked about as a possible instance of self-reflective practice and a future way forward in aligning academic practice to engagement.

Filed under: Indicators Project, self-reflection, Indicators Project, self-reflection, Teaching

Moodle Moot

Just finalising a paper for Moodle Moot this year and looking at what an academic does in their site, adding features adopted over time, then adding use of those features by staff and students to see if this creates involvement/engagement. While the paper is only looking at the old LMS, as CQUniversity has only (Term 1 2010) finalised its move to Moodle, the research is guiding my thinking in the move. It will be interesting, for instance, to replicate the study using Moodle, doing a comparison to Blackboard to see if there are any glaring differences in the data. From the synopsis – Learning Management Systems (LMS) are important tools in the university context. LMS data, whatever LMS a specific university uses, can potentially be used to inform aspects of academic practice to engage students by utilising some common features of the LMS in a way that supports student engagement. Educational research in this study focussed on how an academic interacts with the LMS, how the LMS is used by students, and how these interactions create involvement. Staff interaction with students seems to be one of the key factors in student engagement. This study uses an academic’s approach to teaching + feature adoption + use as an indicator of involvement.

I think this is interesting as it will start to fill in some of the gaps between what an academic states that they do in the LMS, their conception of teaching, and what they actually do, their approach to teaching. While there may be dissonance between what they say they do, and what they actually do, this has to be linked to involvement and engagement. Feature adoption changes over time and some academics have a discussion forum, but they do not use it to contact students then the forum is not an engaged space. Col in this blog post stated, “The participation rate is higher and failure rate is 5% lower for courses where the teaching staff participated in course discussion forums.” This is an important point, if the academic does not approach teaching, adopt features, and get both staff and students using those features then engagement will not happen.

Look at the following figure (Figure 1) detailing academic postings to forums. Why would/should students get engaged?

Figure 1: Course Coordinators posting to forums

Filed under: Indicators Project, self-reflection

Academic Indicators: data mining as reflective practice

Learning Management Systems (LMS) are important tools in the university context and much money, time and resources have been spent in developing, utilizing and maintaining the LMS. These systems assist instructors to administer courses by providing access to content, discussion forums, assignment uploads, grade entry, and other features. Staff facilitating teaching and learning currently use Blackboard for their teaching though there seems to be little use of the LMS to aid in the improvement of pedagogy, or any future planning for teaching and learning. Data, like hit counts, resource utilization, discussion participation, and LMS features that support student engagement, is obtained by applying some simple scripts, data mining, to the LMS backend database. This data is utilized to aid in the reflection of pedagogical practices in alignment with usage statistics. At CQUniversity data could potentially be used to inform aspects of academic practice to engage students by utilising some common features of the LMS in a way that supports student engagement.

There is some evidence (Beer, Jones & Clark, 2009; Malikowski, Thompson & Theis, 2007; Gonzales, 2009) that LMS systems are not being used to their best advantage. They are, in the vast majority of cases, used only as a data repository. Academics use the LMS as a convenient mode of delivering lectures in digital form and then disseminating this to their respective students to make hard copies. If academics had to be categorized in terms of LMS usage, it would be that they are very much content centred though this may be a limitation of the current LMS. Yet, the supposition is, that academics put time and effort into the delivery of their courses, aiming to get the best possible outcome for their students that they can, given the limitations of the management systems we are currently using. Linking LMS usage to engagement, “the time, energy and resources student devote to activities designed to enhance learning at university” (Krause, 2005, p. 3), will aid in reflecting on academic practice and could potentially facilitate engaged teaching, moving the academic from the centre, incorporating real world examples, incorporating reflective methodologies, and shifting the emphasis in teaching from content to dialogue (Hollander, Saltmarsh, & Zlotkowski, 2002).

Researchers seem to look at inherent student qualities that tell part of the story about how students become engaged with the LMS, and with the academic (Ainley, 2004) though there has been research into involvement (Krause, 2005; Goldspink, Winter & Foster, 2008). This study focussed on how an academic interacts with the LMS, how the LMS is used by students, and how these interactions create involvement. Fresen (2007) in researching web-based learning identified staff interaction with students as one of the key factors in student engagement. Dawson and McWilliam (2008, p. 27) also point out that it is not only staff interaction that is crucial, “the quantity of ‘teacher presence’ and quality of ‘teacher presence’ are influencing factors in developing and maintaining student online engagement.”

Reflecting on staff interaction, examining an academic’s approach to teaching and what features they adopt within the LMS will be of some benefit, though these two need to be allied to ‘use’ of the features within the LMS to give some idea of interaction that may explain in some small way staff/student interaction and engagement. In summary, the method taken in this study is that, an academic’s approach to teaching + Feature adoption + Use = involvement (a key factor in student engagement according to Krause, 2005)
Read More (opens as a PDF).

Filed under: Indicators Project, self-reflection, indicators, Indicators Project, Masters, self-reflection

Reflective practice, reconceptualising Gonzales in light of Malikowski et al

One of the aspects of teaching I am engaged in is reflective practice though I am trying to think about this in a new way. In a previous post (see also Col’s post on content and communication)I started to explain that student engagement is important in teaching and learning. Researchers seem to look at inherent student qualities which tell part of the story about engagement (see Ainley (2004) though there has been research into involvement (see Krause (2005), Goldspink et al 2008). From my perspective, I am interested in how I interact with my students to create involvement. Fresen (2007) in researching web-based learning identified staff interaction with students as one of the key factors in student engagement.

Reflecting on staff interaction I think that examining Approaches to teaching and Feature adoption will be of some use, though these two need to be allied to ‘use’ of the tolls within the LMS to give some idea of interaction which may explain in some small way staff/student interaction and engagement.

Approaches to teaching + Feature adoption + Use = interaction (a key factor in student engagement)

One way is to look at features within courses examining the features an academic uses within their course, incorporating Malikowski et al’s (2007) model:

Figure 1: Malikowski et al (2007) feature adoption model

and look at what constitutes a basic approach to teaching though with a focus on Gonzales Dimensions:

Dimensions delimiting approaches to online teaching – (Gonzalez, 2009: p311)

	Informative/individual learning focuses	Communicative/Networked learning focused
Intensity of use	Small range on media and tools used to support learnign tasks and activities (mainly sources of information with small opportunities for interaction and communication)	Wide range of media and tools used to support learning tasks and activities (with emphasis on interaction and communication)
Resources	Web pages with information. Lecture notes. Links to websites.	Web pages with information. Lecture notes. Links to web sites. Discussion boards. Chat. Blogs. Spaces for sharing. Animations. Videos. Still images.
Role of the teacher	Select and present information	Design spaces for sharing and communication. Support the process.
Role of the students	Study individually information provided	Participate in a process of knowledge building

Table 1: Gonzales (2009) Approach to teaching online

Placing an academic in an either content oriented or student oriented perspective and utilising feature adoption will aid in categorising the academic although this does not give an indication of the engagement of the academic nor of the student. What I have done is to reconcetpualize Gonzales (2009) using Malikowski et al (2007) so that within Gonzales a more definite distribution of LMS features are shown (Figure 2 below).

Figure 2: Utilisation of Malikowski’s flowchart in line with Gonzales approaches to teaching

Examining the data with student engagement in mind utilising hit counts, as an indicator of engagement will show that engagement is occurring looking at feature adoption through time.

To test the above model a query was run examining content within courses at CQUniversity. One of the interesting findings from preliminary research into Blackboard (Table 2 below) demonstrates that from 2005 to 2009 (Term 1) the courses with content files has grown 28% over those four years, with the average files per course rising by 18. There appears to be an interesting pattern developing, though more research needs to developed in this area, while files per course has increased the hit counts per student has not risen in the same manner as would be indicated by the data. One possible link here is that students are downloading the files, then printing them out to read at a later date.

Table 2: Content files on Blackboard 2005 – 2009, and average hits per student

In line with the model indicated above (see Figure 2) where Gonzales Dimensions have been modified with Malikowskis’ (2007) feature adoption model, the pattern that emerges from the data is that content far outweighs communicative practice on Blackboard across all courses through 2005 /2009 T1. The huge difference between content and communication can be partially explained by the difference between the ‘hard’ and ‘soft’ sciences but the fact remains that the majority of academics appear to not utilize the communicative networked focused features within the LMS (Figure 3).

Figure 3: Content vs communicative using the adapted model

The interpretation is that there is a huge difference between content and communication/engagement within sites. Another example of this is seen in the following data (Figure 4) where the simple query was run asking the question, “How many academics post to discussion forums?

Figure 4: Course Coordinators posting to forums

The figure is staggering even taking into account that some courses are placeholders for Honours, and Masters subjects. The sheer volume of Course Coordinators that do not post even when provided with a forum is an interesting fact. Of course, more research would have to be done to see if any internal or external factors are involved, but it appears to be very significant in light of student engagement.

Note: For more information on the Indicators project be sure to check out David’s and Col’s respective blogs.

Reference List

Ainley, M. (2004). “What do we know about student motivation and engagement? “ Paper presented at the annual meeting of the Australian Association for Research in Education, Melbourne, November 29-December 2, 2004. Available http:// www.aare.edu.au/04pap/ain04760.pdf.

Fresen, J. (2007). “A taxonomy of factors to promote quality web-supported learning.” International Journal on E-Learning 6(3): 351-362.

Gonzalez, C. (2009). “Conceptions of, and approaches to, teaching online: a study of lecturers teaching postgraduate distance courses.” Higher Education, 57(3): 299-31

Goldspink, C., Winter, P. & Foster, M. (2008). Student Engagement and Quality Pedagogy. URL http://www.earlyyears.sa.edu.au/files/links/Student_Engagement_and_Qua.pdf

Krause, Kerri-Lee. (2005). “Understanding and promoting student engagement in university learning communities.” Centre for the Study of Higher Education, Available http://www.cshe.unimelb.edu.au/pdfs/Stud_eng.pdf.

Malikowski, S., M. Thompson, et al. (2007). “A model for research into course management systems: bridging technology and learning theory.” Journal of Educational Computing Research 36(2): 149-173.

Filed under: Indicators Project, Indicators Project

Indicator’s Presentation

On Monday 27th July, Col Beer and I gave a presentation for the Centre for Workplace Planning,Professional Development & Organisational Learning at CQUniversity. Both of us were slightly nervous though it was time for the project to start to flow to the wider community. The aim is to achieve some sort of reflective practice for academics, and a way for the organisation to guage past and present user behaviour in the LMS. We are also aiming to establish lead indicator’s so that academics, and the wider univeristy community can see how users’ are utilising the LMS in its current form. Slides from the presentation can be seen at Col’s blog.

Filed under: Indicators Project, Indicators Project

My Master’s Project – Expected outcomes

My outcomes are to better understand what academics do online, and how this can be better managed, this research fits into the broad definition of teacher reflective practice.

Learning Management Systems, in the vast majority of cases, are used only as a data repository. I use the LMS as a convenient mode of delivering lectures in hard copy and then disseminate this to my students. If I had to categorize myself I would have to say that I think of myself as being student focused, and I know that I have put a lot time and effort into the delivery of my courses, aiming to get the best possible outcome for my students that I can, given the limitations of the LMS we are currently using. Yet, looking at my courses, I can see that they are content focused and teacher centred.

Gathering the data to inform me of my teaching approach will help in formulating ways to open up dialogue to enhance the student’s learning journey by providing access to resources that create a sense of community where learning is part of the social sphere.

Data mining is only looking at what has occurred from which, one can make inferences about what may occur in the future if nothing changes. Providing access to course content is not enough, students do not engage with the content, nor with other students, and do not engage with the lecturer. Here I am talking about flexible students only, as it is these students who miss the social aspect of interaction that internal student’s access. The base for my involvement with this is Kearsley and Schneiderman’s (1999) paper that involves what the authors’ call;

engaged learning,… all student activities involve active cognitive processes such as creating, problem-solving, reasoning, decision-making, and evaluation … students are intrinsically motivated to learn due to the meaningful nature of the learning environment and activities.

The most important part of this, for me, is ‘the meaningful nature of the learning environment and activities’. If the environment in which students learn is content focused without consideration of the social aspect of learning then it is doomed to fail. This I believe is part of the reason why we do not retain an adequate percentage of our student cohort. Looking at the data within Blackboard will start the process of seeing whether our academics are content focused or student focused, my supposition is that the majority of academics are content focused.

For my project I would like to enhance my own teaching by:

1. Mining the data according to the Dimensions as outlined by Gonzalez (2009)
2. Look at the importance of the social sphere on learning and teaching, for this I will peruse appropriate research, then look at the data as shown by the data above
3. Examine a series of courses using data mining and the Gonzalez’s Dimensions to get some form of baseline
4. Then look at my course using the Dimensions, and seeing if I am content oriented or student oriented.
5. Make changes to the courses to engage students, and create a sense of community within the LMS.
6. In the process of the above to allow peers to review what I am doing, and to see if the study proposed has a wider context within the university context.

Filed under: Indicators Project, Indicators Project, Masters