How good are you at programming? A CEFR-like approach to measure programming proficiency

The table

Also available in PDF form.

The online test

A web application is available to assess your own skills according to this table for one or more programming languages.

How to use this table

The table characterizes the proficiency level (columns) of programmers of a particular programming language in the context of different programming activities (rows).

This table is inspired by the CEFR table of the same name, for assessing proficiency in natural languages. Like the CEFR, this table divides learners into three broad level divisions: “Basic user” (A), “Independent user” (B) and “Proficient user” (C). The broad divisions are each further divided in two levels (A1, A2, B1, B2, C1, C2) that correspond to testable milestones in language acquisition.

This table can be used in different ways, for example:

• row by row, to assess one’s own level per activity (different skill levels for different activities are possible);
• column by column from left to right, to determine one’s own minimum level for a programming language (the rightmost level where all requirements in the column and all columns to the left are matched);
• column by column from right to left, to determine one’s most developed skill (the rightmost level where any requirement in the column is matched);
• language per language, to assess one’s own relative proficiency in different programming languages.

Possible applications

• to track one’s own progress while learning how to program;

  for example: this year, I transitioned from A2 to B2 in C++. For Java, I am B1 for understanding but still A2 for writing.

• to advertise the educational goals of a programming course;

  for example: this Java introductory course will bring you to level A1 or A2 for all activities.

• to advertise one’s own skillset to peers or prospective employers;

  for example: I am C1 in Python, B2 in O’Caml and A2 in Haskell.

• to set basic level requirements for courses or professional activities:

  for example: This course requires A2 proficiency with a language in the C family.

• to select a programming course that best matches one’s skill level;

  for example: My current level is A2 but this course requires B1, so I will need some extra work before starting.

• to coordinate the teaching objectives of successive programming courses in a curriculum;

  for example: Our introductory course brings students to A2 in Java, but our follow-up program requires B1 or B2, so we need to propose a supplementary course for that level.

Design methodology

The table was designed following the CEFR methodology:

First, recognize the different modalities (production, reception, interaction, mediation). Then group and abstract activities in each modality by clusters that share similar motivations and actors. The resulting set becomes the rows in the matrix.

The requirements for “A” levels are then phrased to identify users that can perform language acts under supervision or under dedicated/personalized guidance from peers.

The requirements for “B” levels are then phrased to identify users that can perform language acts without supervision or guidance from peers, or with minimal effort from peers.

The requirements for “C” levels are then phrased to identify users who are fully independent and who can demonstrate skills corresponding to a high expertise level according to the majority of other users of the language.

References

• Common European Framework of Reference for Languages: Learning, Teaching, Assessment (CEFR). Language Policy Unit, Strasbourg. Available from: http://www.coe.int/lang-CEFR
• Wikipedia: Common European Framework of Reference for Languages.

Acknowledgements

The initial idea of an assessment matrix appeared during discussions with Merijn Verstraaten about average programmer skills in online fora. The concrete form as presented here was further developed thanks to inspiration and suggestion from Philip Hölzenspies, Jan Kuper, Cosmin Dumitru, Taddeüs Kroes, and many others.