Beer, C. Clark, K., & Jones, D. (2010). Indicators of engagement. In C.H. Steel, M.J. Keppell, P. Gerbic & S. Housego (Eds.), Curriculum, technology & transformation for an unknown future. Proceedings ascilite Sydney 2010 (pp.75-86). http://www.ascilite.org.au/conferences/sydney10/Ascilite conference proceedings 2010/Beer-full.pdf

Clark, K., Beer, C. & Jones, D. (2010). Academic involvement with the LMS: An exploratory study. In C.H. Steel, M.J. Keppell, P. Gerbic & S. Housego (Eds.), Curriculum, technology & transformation for an unknown future. Proceedings ascilite Sydney 2010 (pp.487-496). http://www.ascilite.org.au/conferences/sydney10/Ascilite conference proceedings 2010/Kenclark-full.pdf

Monday 6th December 2010.

The conference was opened with a keynote from Professor Jan Herrington (UTS) on Authentic Learning and emerging technologies, followed by a plenary session on Blackboard.

After the plenary session I listened to some fascinating papers on LMS system analysis and Blended Learning environments. The afternoon session was spent finalizing the first of the papers to be delivered on Tuesday and Wednesday.

Tuesday 7th December 2009.

The morning keynote was by Dr Lev Gonick who spoke of a very high speed interconnectedness in downtown Cleveland, and the difficulties that surround that project, but he also spoke of the possibilities of these developments.

This followed by the invited speaker, Prof Tom Reeves, who spoke on the challenges of online education, and blended learning environments.

Our first paper, presented by Col, was after the lunch session and was on indicators of engagement, which was well received, and originated some useful discussion.

The rest of the day was spent going to other presentations, and we broke early so that we could get ready for the conference dinner.

Wednesday 8th December 2009.

Whilst somewhat dry from the night before, the morning sessions in my breakout group were interesting and the morning was spent getting ready for the second presentation, due after the morning tea break. At this session I spoke on the opportunities for reflective practice that exists in academic analytics.

The conference was interesting, though there is still a lot of work to do with academic analytics, and it was very good to catch up once again with Shane Dawson who has agreed to be a supervisor for my PhD.  And as Shane said, I need to read, “what is your research question”, and basically write, even if it is crap.  I agree with this, it is the art of writing, nit looking for the ‘gems’ that make a successful writer.

Overall, it was a good conference but I was glad to get home

cheers

Ken

Abstract:
There is growing interest in the use of academic analytics however most of the reported work is being done at the level of institutions, and groupings of courses within those institutions. This study is an exploratory case study aimed at analyzing an academics’ involvement with the Learning Management System, the student’s involvement with the LMS, and the links between the LMS, the academic, and the students.

To read more. (Opens as a pdf)

The presentation for ascilite 2010:

I think this is interesting as it will start to fill in some of the gaps between what an academic states that they do in the LMS, their conception of teaching, and what they actually do, their approach to teaching. While there may be dissonance between what they say they do, and what they actually do, this has to be linked to involvement and engagement. Feature adoption changes over time and some academics have a discussion forum, but they do not use it to contact students then the forum is not an engaged space. Col in this blog post stated, “The participation rate is higher and failure rate is 5% lower for courses where the teaching staff participated in course discussion forums.” This is an important point, if the academic does not approach teaching, adopt features, and get both staff and students using those features then engagement will not happen.

Look at the following figure (Figure 1) detailing academic postings to forums. Why would/should students get engaged?

Figure 1: Course Coordinators posting to forums

There is some evidence (Beer, Jones & Clark, 2009; Malikowski, Thompson & Theis, 2007; Gonzales, 2009) that LMS systems are not being used to their best advantage. They are, in the vast majority of cases, used only as a data repository. Academics use the LMS as a convenient mode of delivering lectures in digital form and then disseminating this to their respective students to make hard copies. If academics had to be categorized in terms of LMS usage, it would be that they are very much content centred though this may be a limitation of the current LMS. Yet, the supposition is, that academics put time and effort into the delivery of their courses, aiming to get the best possible outcome for their students that they can, given the limitations of the management systems we are currently using. Linking LMS usage to engagement, “the time, energy and resources student devote to activities designed to enhance learning at university” (Krause, 2005, p. 3), will aid in reflecting on academic practice and could potentially facilitate engaged teaching, moving the academic from the centre, incorporating real world examples, incorporating reflective methodologies, and shifting the emphasis in teaching from content to dialogue (Hollander, Saltmarsh, & Zlotkowski, 2002).

Researchers seem to look at inherent student qualities that tell part of the story about how students become engaged with the LMS, and with the academic (Ainley, 2004) though there has been research into involvement (Krause, 2005; Goldspink, Winter & Foster, 2008). This study focussed on how an academic interacts with the LMS, how the LMS is used by students, and how these interactions create involvement. Fresen (2007) in researching web-based learning identified staff interaction with students as one of the key factors in student engagement. Dawson and McWilliam (2008, p. 27) also point out that it is not only staff interaction that is crucial, “the quantity of ‘teacher presence’ and quality of ‘teacher presence’ are influencing factors in developing and maintaining student online engagement.”

Reflecting on staff interaction, examining an academic’s approach to teaching and what features they adopt within the LMS will be of some benefit, though these two need to be allied to ‘use’ of the features within the LMS to give some idea of interaction that may explain in some small way staff/student interaction and engagement. In summary, the method taken in this study is that, an academic’s approach to teaching + Feature adoption + Use = involvement (a key factor in student engagement according to Krause, 2005)
Read More (opens as a PDF).

After this I walked back to my Hotel which is about 20 minutes walk up the hill.  Then when Col and David arrived we went back down the hill for a meal, and a couple of drinks, then another walk up the hill, not good on the old feet.

One of the first things we realised was that it did not work.  The basic premise that we understood at that time was that students were tech savvy, and therefore, they would use RSS feeds and integrate them into their life.  Alas, such is not the case.  “Build it and they will come”, or “Gen Y is tech savvy’ is not quite true.  One or two students used it, but the majority did not as they saw no use for such technology, as they use technology for play.  As one student said to me, “I use a mobile phone, an IPod, and Facebook; other things [by this I think she meant technology] do not interest me.”

The idea is still good though, and I am exploring ways that staff and industry peers can use this site to facilitate some form of engagement with each other.  Perhaps there is some way to utilise the feeds in Facebook?  Will have to explore this option.

Reflecting on staff interaction I think that examining Approaches to teaching and Feature adoption will be of some use, though these two need to be allied to ‘use’ of the tolls within the LMS to give some idea of interaction which may explain in some small way staff/student interaction and engagement.

Approaches to teaching + Feature adoption + Use = interaction (a key factor in student engagement)

One way is to look at features within courses examining the features an academic uses within their course, incorporating Malikowski et al’s (2007) model:

Figure 1: Malikowski et al (2007) feature adoption model

and look at what constitutes a basic approach to teaching though with a focus on Gonzales Dimensions:

Table 1: Gonzales (2009) Approach to teaching online

Placing an academic in an either content oriented or student oriented perspective and utilising feature adoption will aid in categorising the academic although this does not give an indication of the engagement of the academic nor of the student. What I have done is to reconcetpualize Gonzales (2009) using Malikowski et al (2007) so that within Gonzales a more definite distribution of LMS features are shown (Figure 2 below).

Figure 2: Utilisation of Malikowski’s flowchart in line with Gonzales approaches to teaching

Examining the data with student engagement in mind utilising hit counts, as an indicator of engagement will show that engagement is occurring looking at feature adoption through time.

To test the above model a query was run examining content within courses at CQUniversity. One of the interesting findings from preliminary research into Blackboard (Table 2 below) demonstrates that from 2005 to 2009 (Term 1) the courses with content files has grown 28% over those four years, with the average files per course rising by 18. There appears to be an interesting pattern developing, though more research needs to developed in this area, while files per course has increased the hit counts per student has not risen in the same manner as would be indicated by the data. One possible link here is that students are downloading the files, then printing them out to read at a later date.

Table 2: Content files on Blackboard 2005 – 2009, and average hits per student

In line with the model indicated above (see Figure 2) where Gonzales Dimensions have been modified with Malikowskis’ (2007) feature adoption model, the pattern that emerges from the data is that content far outweighs communicative practice on Blackboard across all courses through 2005 /2009 T1. The huge difference between content and communication can be partially explained by the difference between the ‘hard’ and ‘soft’ sciences but the fact remains that the majority of academics appear to not utilize the communicative networked focused features within the LMS (Figure 3).

Figure 3: Content vs communicative using the adapted model

The interpretation is that there is a huge difference between content and communication/engagement within sites. Another example of this is seen in the following data (Figure 4) where the simple query was run asking the question, “How many academics post to discussion forums?

Figure 4: Course Coordinators posting to forums

The figure is staggering even taking into account that some courses are placeholders for Honours, and Masters subjects. The sheer volume of Course Coordinators that do not post even when provided with a forum is an interesting fact. Of course, more research would have to be done to see if any internal or external factors are involved, but it appears to be very significant in light of student engagement.

Note: For more information on the Indicators project be sure to check out David’s and Col’s respective blogs.

Reference List

Ainley, M. (2004). “What do we know about student motivation and engagement? “ Paper presented at the annual meeting of the Australian Association for Research in Education, Melbourne, November 29-December 2, 2004. Available http:// www.aare.edu.au/04pap/ain04760.pdf.

Fresen, J. (2007). “A taxonomy of factors to promote quality web-supported learning.” International Journal on E-Learning 6(3): 351-362.

Gonzalez, C. (2009). “Conceptions of, and approaches to, teaching online: a study of lecturers teaching postgraduate distance courses.” Higher Education, 57(3): 299-31

Goldspink, C., Winter, P. & Foster, M. (2008). Student Engagement and Quality Pedagogy. URL http://www.earlyyears.sa.edu.au/files/links/Student_Engagement_and_Qua.pdf

Krause, Kerri-Lee. (2005). “Understanding and promoting student engagement in university learning communities.” Centre for the Study of Higher Education, Available http://www.cshe.unimelb.edu.au/pdfs/Stud_eng.pdf.

Malikowski, S., M. Thompson, et al. (2007). “A model for research into course management systems: bridging technology and learning theory.” Journal of Educational Computing Research 36(2): 149-173.

Learning Management Systems, in the vast majority of cases, are used only as a data repository. I use the LMS as a convenient mode of delivering lectures in hard copy and then disseminate this to my students. If I had to categorize myself I would have to say that I think of myself as being student focused, and I know that I have put a lot time and effort into the delivery of my courses, aiming to get the best possible outcome for my students that I can, given the limitations of the LMS we are currently using. Yet, looking at my courses, I can see that they are content focused and teacher centred.

Gathering the data to inform me of my teaching approach will help in formulating ways to open up dialogue to enhance the student’s learning journey by providing access to resources that create a sense of community where learning is part of the social sphere.

Data mining is only looking at what has occurred from which, one can make inferences about what may occur in the future if nothing changes. Providing access to course content is not enough, students do not engage with the content, nor with other students, and do not engage with the lecturer. Here I am talking about flexible students only, as it is these students who miss the social aspect of interaction that internal student’s access. The base for my involvement with this is Kearsley and Schneiderman’s (1999) paper that involves what the authors’ call;

engaged learning,… all student activities involve active cognitive processes such as creating, problem-solving, reasoning, decision-making, and evaluation … students are intrinsically motivated to learn due to the meaningful nature of the learning environment and activities.

The most important part of this, for me, is ‘the meaningful nature of the learning environment and activities’. If the environment in which students learn is content focused without consideration of the social aspect of learning then it is doomed to fail. This I believe is part of the reason why we do not retain an adequate percentage of our student cohort. Looking at the data within Blackboard will start the process of seeing whether our academics are content focused or student focused, my supposition is that the majority of academics are content focused.

For my project I would like to enhance my own teaching by:

1. Mining the data according to the Dimensions as outlined by Gonzalez (2009)
2. Look at the importance of the social sphere on learning and teaching, for this I will peruse appropriate research, then look at the data as shown by the data above
3. Examine a series of courses using data mining and the Gonzalez’s Dimensions to get some form of baseline
4. Then look at my course using the Dimensions, and seeing if I am content oriented or student oriented.
5. Make changes to the courses to engage students, and create a sense of community within the LMS.
6. In the process of the above to allow peers to review what I am doing, and to see if the study proposed has a wider context within the university context.