O'Really?

March 27, 2023

Join us to discuss using AI to solve simple programming problems on Monday 3rd April at 2pm BST

Filed under: sigcse — Duncan Hull @ 10:14 am
Tags: , ,
CC licensed pilot image from flaticon.com

Maybe you wrote that code and maybe you didn’t. If Artificial Intelligence helped you, such as the OpenAI Codex in GitHub Copilot, how did it solve your problem? How much did AI help or hinder your solution? Join us to discuss a paper by Michel Wermelinger from the Open University published in the SIGCSE technical symposium earlier this month in Toronto on this very topic. [1] We’ll be joined by Michel who will present a lightning talk to kick-off our discussion. Here’s the abstract of his paper:

The teaching and assessment of introductory programming involves writing code that solves a problem described by text. Previous research found that OpenAI’s Codex, a natural language machine learning model trained on billions of lines of code, performs well on many programming problems, often generating correct and readable Python code. GitHub’s version of Codex, Copilot, is freely available to students. This raises pedagogic and academic integrity concerns. Educators need to know what Copilot is capable of, in order to adapt their teaching to AI-powered programming assistants. Previous research evaluated the most performant Codex model quantitatively, e.g. how many problems have at least one correct suggestion that passes all tests. Here I evaluate Copilot instead, to see if and how it differs from Codex, and look qualitatively at the generated suggestions, to understand the limitations of Copilot. I also report on the experience of using Copilot for other activities asked of students in programming courses: explaining code, generating tests and fixing bugs. The paper concludes with a discussion of the implications of the observed capabilities for the teaching of programming.

All welcome, as usual we’ll be meeting on zoom, details at sigcse.cs.manchester.ac.uk/join-us

References

  1. Michel Wermelinger (2023) Using GitHub Copilot to Solve Simple Programming Problems in Proceedings of the 54th ACM Technical Symposium on Computer Science Education Pages SIGCSE 2023 page 172–178 DOI: 10.1145/3545945.3569830

July 4, 2022

Join us to discuss the implications of the Open AI codex on introductory programming Monday 4th July at 2pm BST

Automatic code generators have been with us a while, but how do modern AI powered bots perform on introductory programming assignments? Join us to discuss the implications of the OpenAI Codex on introductory programming courses on Monday 4th July at 2pm BST. We’ll be discussing a paper by James Finnie-Ansley, Paul Denny, Brett A. Becker, Andrew Luxton-Reilly and James Prather [1] for our monthly SIGCSE journal club meetup on zoom. Here is the abstract:

Recent advances in artificial intelligence have been driven by an exponential growth in digitised data. Natural language processing, in particular, has been transformed by machine learning models such as OpenAI’s GPT-3 which generates human-like text so realistic that its developers have warned of the dangers of its misuse. In recent months OpenAI released Codex, a new deep learning model trained on Python code from more than 50 million GitHub repositories. Provided with a natural language description of a programming problem as input, Codex generates solution code as output. It can also explain (in English) input code, translate code between programming languages, and more. In this work, we explore how Codex performs on typical introductory programming problems. We report its performance on real questions taken from introductory programming exams and compare it to results from students who took these same exams under normal conditions, demonstrating that Codex outscores most students. We then explore how Codex handles subtle variations in problem wording using several published variants of the well-known “Rainfall Problem” along with one unpublished variant we have used in our teaching. We find the model passes many test cases for all variants. We also explore how much variation there is in the Codex generated solutions, observing that an identical input prompt frequently leads to very different solutions in terms of algorithmic approach and code length. Finally, we discuss the implications that such technology will have for computing education as it continues to evolve, including both challenges and opportunities. (see accompanying slides)

All welcome, details at sigcse.cs.manchester.ac.uk/join-us. Thanks to Jim Paterson at Glasgow Caledonian University for nominating this months paper.

References

  1. James Finnie-Ansley, Paul Denny, Brett A. Becker, Andrew Luxton-Reilly, James Prather (2022) The Robots Are Coming: Exploring the Implications of OpenAI Codex on Introductory Programming ACE ’22: Australasian Computing Education Conference Pages 10–19 DOI:10.1145/3511861.3511863

April 28, 2020

Join us to discuss learning programming languages: Monday 4th May at 11am #sigcsejclub

Filed under: education,engineering,Uncategorized — Duncan Hull @ 10:17 am
Tags: , , , ,

Hieroglyphs_from_the_tomb_of_Seti_I

Hieroglyphs from the tomb of Seti I, by Jon Bodsworth via Wikimedia Commons and the Egypt archive

ACM SIGCSE Journal Club returns Monday 4th May at 11am. The paper we’re discussing this month is “Relating Natural Language Aptitude to Individual Differences in Learning Programming Languages” by Chantel Prat et al published in Scientific Reports. [1] Here’s the abstract:

This experiment employed an individual differences approach to test the hypothesis that learning modern programming languages resembles second “natural” language learning in adulthood. Behavioral and neural (resting-state EEG) indices of language aptitude were used along with numeracy and fluid cognitive measures (e.g., fluid reasoning, working memory, inhibitory control) as predictors. Rate of learning, programming accuracy, and post-test declarative knowledge were used as outcome measures in 36 individuals who participated in ten 45-minute Python training sessions. The resulting models explained 50–72% of the variance in learning outcomes, with language aptitude measures explaining significant variance in each outcome even when the other factors competed for variance. Across outcome variables, fluid reasoning and working-memory capacity explained 34% of the variance, followed by language aptitude (17%), resting-state EEG power in beta and low-gamma bands (10%), and numeracy (2%). These results provide a novel framework for understanding programming aptitude, suggesting that the importance of numeracy may be overestimated in modern programming education environments.

The paper describes an experiment which investigates the relationship between learning natural languages and programming languages and draws some interesting conclusions that provide some good discussion points. Does being good at learning natural languages like English make you good at learning programming language like Python? Do linguists make good coders? We’ll be meeting on Zoom, details will be sent to anyone who registers at sigman2.eventbrite.co.uk

References

  1. Prat, C.S., Madhyastha, T.M., Mottarella, M.J. et al. (2020) Relating Natural Language Aptitude to Individual Differences in Learning Programming Languages. Scientific Reports 10, 3817 (2020). DOI:10.1038/s41598-020-60661-8

 

June 23, 2017

Nine ideas for teaching Computing at School from the 2017 CAS conference

CAS

Delegates at the Computing at School conference 2017 #CASConf17 answering diagnostic questions, picture by Miles Berry.

The Computing At School (CAS) conference is an annual event for educators, mostly primary and secondary school teachers from the public and private sector in the UK. Now in its ninth year, it attracts over 300 delegates from across the UK and beyond to the University of Birmingham, see the brochure for details. One of the purposes of the conference is to give teachers new ideas to use in their classrooms to teach Computer Science and Computational Thinking. I went along for my first time (*blushes*) seeking ideas to use in an after school Code Club (ages 7-10) I’ve been running for a few years and also for approaches that undergraduate students in Computer Science (age 20+) at the University of Manchester could use in their final year Computer Science Education projects. So here are nine ideas (in random brain dump order) I’ll be putting to immediate use in clubs, classrooms, labs and lecture theatres:

  1. Linda Liukas demonstrated some intriguing ideas from her children’s books and HelloRuby.com that are based on Montessori education. I shall be trying some of these out (particularly the storytelling stuff) at code club to keep girls involved
  2. Sue Sentance and Neil Brown from King’s College London gave an overview of some current research in pedagogy.  They discussed research questions that can be tackled in the classroom like (for example) do learners make more progress using visual programming languages (like Scratch and Blockly) or traditional text-based languages (like Python and Java etc)? Many of these research questions would make good projects for undergraduate students to investigate in secondary schools, see research on frame based editors, for example.
  3. Michel Wermelinger from the Open University demonstrated using iPython notebooks for teaching data literacy at the Urban Data School. Although I’m familiar with iPython, it had never occurred to me to actually use iPython in school for teaching. It is a no-brainer, when you think about it, even for primary, because you have your code, inputs and outputs all in one window, and can step through code execution instead of (or as well as) using more conventional tools like Trinket, Thonny or IDLE. Data literacy is fun to teach.
  4. Miles Berry from the University of Roehampton demonstrated Diagnostic Questions in Project Quantum. These are a collection of high quality quizzes to use interactively for example as hinge questions, where teaching is adapted depending on answers given, like this multiple choice question:
    Consider the following Python code:
    
    a = 20
    b = 10
    a = b
    
    What are the values of a and b?
    
    A: a = 10, b = 10
    B: a = 20, b = 20
    C: a = 30, b = 10
    D: a = 10, b = 20
    

    You’ll have to try these five questions to check your answer. The useful thing here is that DiagnosticQuestions.com (the platform on which this is built) allows you to see lots of responses, for example each answer (A, B, C or D) above was selected by 25% of participants. You can also view explanations which illuminate common misconceptions (e.g. the classic mistake of confusing assignment with equality) as well as providing a bank of free questions for use in the classroom.

  5. Mark Guzdial from GeorgiaTech discussed using learning sciences to improve computing teaching. He demonstrated predictive questions (e.g. ask students What do you think will happen when we run this code? before actually executing it) alongside what he called subgoal labelling. These are simple ideas (with proven benefits) that can be put to use immediately. I’ll also be trying the Live Coding (with Sonic Pi) and Media Computation he demonstrated asap.
  6. Laurence Rogers demonstrated Insight: Mr. Bit  this looks like a good app for using BBC microbits in the classroom, connected to a range of sensors, provided you’ve got access to iPads.
  7. A copy of Hello World magazine was in the conference bag. The summer 2017 issue has an unusual article from Ian Benson from Kingston University and Jenny Cane describing their use of the Haskell programming language to teach 5-7 year olds to reason symbolically and learn algebra before arithmetic with help from Cuisenaire rods. The Scratch Maths project at University College London are doing similar things, building mathematical knowledge using Scratch, rather than Haskell. These are experimental ideas you could try out on unsuspecting (junior) family members.
  8. Lee Goss from Barefoot Computing, described the free CPD for primary school teachers on offer from BT. I’ve signed up and hope to plug some of the shortcomings in the Code Club Curriculum.
  9. Richard Jarvis demonstrated appJar, a handy Python library for teaching Graphical User Interfaces (GUIs). That’s Jar as in Jarvis and Jam, not JAR as in Java ARchive BTW. I’ve not tried GUIs at code club yet, but appJar looks like a good way to do it.

There were lots more people and projects at the conference not mentioned here including tonnes of workshops. If you’re interested in any of the above, the CAS conference will be back in 2018. Despite the challenging problems faced by Computer Science at GCSE level, it was reassuring and inspiring to meet some members of the vibrant, diverse and friendly community pushing the boundaries of computing in schools across the United Kingdom. Thanks again to everyone at CAS for putting on another great event, I will definitely consider attending next year and maybe you should too.

March 15, 2013

Creating with the Raspberry Pi vs. Consuming Apple Pie at the Manchester Raspberry Jamboree

MiniGirlGeek

Thirteen year old Amy Mather aka @MiniGirlGeek steals the show at Manchester Raspberry Jam 2013

Last Saturday, the first ever Raspberry Jamboree rolled into town, organised by the unstoppable force of nature that is Alan O’Donohoe (aka @teknoteacher). The jamboree looked at the educational value of the Raspberry Pi (a $25 computer) one year on from its launch on the the 29th February 2012. Here are some brief and incomplete notes on some of the things that happened in the main room, aka ‘Jamboree Central’. The workshops and other events have been covered by Jason Barnett @boeeerb.

A key feature of the Raspberry Pi foundation (and the Jamboree) was neatly summed up by Paul Beech (aka @guru) who compared the Raspberry Pi to various Apple iThings. Paul’s view is that when it comes to computing, Apple gives you a “sandy beach, sunbed and cocktail” to passively consume digital content with while the Raspberry Pi gives you a “desert, knife and a bottle” to actively create new things (see his tweet below).

Consuming Apple Pie on a sandy beach, with a sunbed and a cocktail

Engineering evangelist Rob Bishop used Apple Inc. to illustrate what the Raspberry Pi is about in his talk ‘one year on‘. Rob pointed out that a huge amount of effort at Apple Inc. is put into making Computing invisible and seamless. This is great if you’re consuming content on your iPad or iPhone, and what many users want – easy to use, with all the nasty internal gubbins tucked away, out of sight. This is tasty Californian Apple Pie, which many of consume in large amounts.

However, invisible computing is a problem for education, because it is difficult to demonstrate the Wonders of Computer Science (Brian Cox’s next TV series) with a device like the iPad.  Many of the internals of modern devices are completely inaccessible, and it’s non-trivial for budding young engineers to build anything very interesting with it particularly quickly.

In contrast, the Raspberry Pi can be challenging to setup, just getting the Operating System up and running isn’t always straightforward. However, there’s a ton of interesting stuff you can build with it: Nifty robotics, bionic bird boxes, musical hackery, twittering chickens, live train departure boards, internet radiossinging jelly babies and loads of other pideas. Try doing that with your iPad…

Creating with Raspberry Pi in the desert, using a knife and a bottle

Most of the jamboree focussed not on Apple but on the things that can be created with Raspberry Pi: the What and Why and When And How and Where and Who with keynotes from Steve Furber [1] and talks and panel sessions from:

A highlight of the jamboree was the closing keynote given by the thirteen year old Mini Girl Geek on what she’s been doing with her Raspberry Pi. MiniGirlGeek (aka Amy Mather pictured above) stole the show with her demo implementations of Conway’s Game of Life in Python. [update: see video below]

What’s interesting is that Conway’s Game of Life is used as an exercise for first year undergraduates in Computer Science at the University of Cambridge. So it’s great to see teenagers mastering the “knife” of Raspberry Pi, and reminds us that Raspberry Pi is no “sunbed and cocktail” but with a little patience, ambition and talent there’s plenty to capture the imagination of young people about Computing.

References

  1. Steve Furber et al (2012). Computing in Schools: Shut down or restart? Royal Society Report

Blog at WordPress.com.