Community certification

Over the last few years I’ve had several projects where I had to interview people to get involved on Flash/Flex/ActionScript projects — typically what happens is the client wants some sort of assurance that “nothing bad will happen” and somehow there is this perception that only getting people on board that have passed a certification exam helps with this.

From my experience that is not the case and I have serious doubts about the whole concept of a couple dozen multiple choice questions to determine someone’s expertise in a particular product. At best it documents that they know the theory.

Is software certification worth something in the real-world and is it worth you spending your money on? I’ve seen a lot of qualified people rejected for projects because they didn’t have product certification and on the other hand you have people that did get certified and couldn’t script a simple animation without resorting to the documentation.

I would like to see this changed.

For a while now I’ve been thinking about the possibility to set up an open community run program that documents real-world expertise. This concept is not too far from what JavaBlackBelt is doing.

The basic concept

  • Free access to certification exams

    Cost shouldn’t be a barrier to entry for deciding to take a certification exam, just think about the amount of skilled students and people starting out as developers that are not in a position to afford this and whose skills go unnoticed.

  • Recognize specific areas of expertise

    Everyone has specific areas of expertise, even within one product. When you take a certification exam you get the breakdown in your results print out but they’re not published.

  • Peer review of your code

    Multiple choice questions alone are not an adequate way to test proficiency using a technology. What I’d propose is requiring candidates to submit some open source code snippets, open to the public and reviewed based on a set of evaluation criteria by a group of peers.

  • Community visibility

    Once somebody get certified, there needs to be a public resource to get that published on. It should list your product certification status, main areas of expertise as well your submitted open source code snippets.

I’d ideally like to tie this into a mentorship program — where beginning developers can be assigned a community member to help them build their skill level.

Just to be clear, I don’t see this necessarily replacing traditional software certification but as an additional way to demonstrate your real-world expertise and community involvement. This concept can be extended from just software certification to more specific skills like e.g. Papervision3D, PureMVC, etc.

I would love to hear your thoughts on this, if there’s enough interest I’d be happy to set up this community certification initiative and start recruiting some volunteers.


6 thoughts on “Community certification

  1. Mark Young says:

    Hi Peter

    like your comments and agree with some of the things you say. We’d been discusing this at Academy Class as well so think its a good idea but alow me to play devils advocate.

    Firstly who would the certification exams be for?
    If its for seasoned developers then shouldn’t their past references and work portfolio be enough so a certification isn’t really needed? I’ve spoken with a few recruiters in my time and asked them if certification in their field was important to the employers and they have said no. When it comes to quite high level stuff then its al about the past experiences and projects they were involved in. That’s what matters most… although certification is a good differentiation for the CV.

    If the exams were aimed at students who may not have had the work experience yet required for certain job roles then surely something like the Adobe Certified Expert exams (which are actually pretty tough) would prove they are more than competent with the software and so be a good benchmark to get their first job. We have students coming into Academy Class for training who have just finished their degrees and a lot of them still have a lot to learn about the actual technology they are going to be using in the real world. A certification bny the manufacturer is at least a target for them to achieve and they will need to know their stuff to pass the exams.

    If a community certification is set up is set up then there’s a few things to think about.
    How do you get it recognised by industry?
    Who sets and marks and awards the certification and who deems when someone is good enough to pass or has failed?
    Where is the certification carried out (physical location)
    There are so many communities in the different geographical locations. What makes one community better than the other to be able to award certifications to those who apply for the award.
    How do you police certification and what measures would be put in place to stop corruption and self interest. Surely individuals could be bribed to allow someone to pass an exam.
    And who would have the time to actually manage the whole process, promote, organise and all that stuff.

    The way the certified adobe exams are done now are at a facility where you need to book the exam. There are exam conditions (quite room, no cheat sheets brought into class), identification is needed to be able to sit the exam (so no getting someone to pretend they are you and pass the exam). The prometric vue testing centres are set up to do make sure everything is fair and there are no double standards.

    All it takes is one mistake, one corrupt member who accepts a bribe to pass someone, one other community / usergroup to bad mouth the certification and then its game over. It could be months / years of hard work down the drain. A certification I believe would still need backing of a manufacturer or maybe even a training centre (although thats self interest from me) to make something like this happen. It would really need someone strong and dedicated with a lot of spare time on their hands to carry this project through or it will fall to the wayside. Maybe what the manufacturers should be doing is actually promoting the certifications more to industry, giving them more value, or better still working with the community to develop better ways of testing. But testing doesn’t sell boxes of software.

    thats my tuppence worth. I think its definitely worth further discussion and we would like to be involved in a project like this



  2. DannyT says:

    I really like the idea of this, although I’m skeptical about levels of commitment that will be given by the “experts”. With the best will in the world I think there are only very few who would be dedicated or even able to offer their time for little or no gain to themselves other than feel-good factor.

    Perhaps if there was some low-time investment that could be passed around easily from expert-to-expert then it could be good. For example if an existing expert anonymously screen-shares with someone trying to gain accreditation for 20-30 minutes whilst they (the candidate) work on a related technical task, the expert then rates that candidate out of 10 in their ability for that task and that’s it. The candidate could perhaps do this with three experts on three different tasks and if their total score is say 25 or higher they get ‘upgraded’ to expert who then has to commit to doing the same for the next candidate.

    Or even once you’ve achieved the score you have to do two assessments to finalise your accreditation or something.

    Doing something like this will keep the ball rolling rather than depend on a select few to commit their time who, once they get busy, could let the whole scheme fall to the wayside as happens with so many other great initiatives.

  3. wonderwhy-er says:

    Interesting problem you are trying to tackle. It seems you still want to go a way of exams in some way… Not shore that that represents real world well… Plus as some have said here it will be hard to make reputation for them.

    I think I would have went other way… Something like making site for portfolios that includes expertise rankings. For example I made some kind of job and added this work to that site. If I can I also added code. If I can I shared some info on client and may be made client to write his review. Then other community users can check the work and write their comments on that portfolio item judging various parameters like performance, technical complexity, some specific library usage, interface and interactivity quality.

    Again expert and client credibility is the problem. Tough may be it can be solved in same way say eBay solves client and seller credibility trough reputations systems and constant user reputation and item reputation revision.

    I think this way it will be more of real world expertise evaluation then exams or even specially written for that purpose code.

  4. I personally have some thoughts about who would create the certification projects or judge them. This could become a select club of maybe a dozen members that do anything to keep themselves “ahead of the rest” by doing whatever they can to flunk people. SO yeah, I’m kind of skeptical about this.

    A couple of months ago I heard from an Adobe employee from Florida, who is also an instructor and one of the guys that actually writes the courses, that they are planning to change the ACE process. Mainly they want to avoid having “experts” that are just good students, but don’t know how things work in real life. Apparently they are working on something like you are suggesting, Peter: finishing a certain project by a certain deadline and making sure it works ;-).

    So, yes to that: I’m definitely for that type of examination; even if you’d have to defend the results to an independent jury. This could mean you really know something. And it’s not because you use the livedocs every day, that you are not a good programmer.

    However, when you talk about certification as an ACI, that would be a whole other thing all together. Then you really have to know your stuff, have didactical and educational skills, as well as an impressive track record in the technology. I’d hate it when they put an official instructor in front of me that only know the theory and teaches “ex cathedra”…

    But anyway, making experts more visible would make it more interesting for people to go through the certification process. Especially if the businesses would really give a damn about these things, ’cause I personally get the feeling nobody knows what an ACE or ACI is :-S

  5. Hi Peter,

    Excellent idea, sounds great… I can see that the domains flashblackbelt and flexblackbelt are already registered, so I assume others are thinking in same terms.

    Are you continuing with this ?

    // Peter

  6. Peter says:

    Thanks for the feedback guys, thats excellent! Its obviously not an easy thing to set up but think its worth exploring the idea.

    A couple of points:

    – I see this more as an additional level of certification, establishing your expertise applying your product skills to real-world projects. As such if we go down the exam route as part of the certification process, questions won’t be multiple choice but rather something like “how would you structure an application that needs to hold data loaded in from a webservice and update the UI once changes have been detected”.

    – Don’t see any practical way how these certification tests can be done on location, and if there were that would be another barrier to entry for those with no available testing centre in their immediate area.

    I’m seeing this happening as an online system, requiring candidates to submit code that will be open to the public with an additional rating system. Once people are certified, people should be able to give a thumbs up / thumbs down — providing some details to back that up. If a certified individual goes below a certain trust threshold we’d require him to go through the certification process again and put his status on hold.

    – Anyone who has demonstrated expertise in the product should be able to join to assess candidates. The exam reviews should be done anonymously and by three different experts, after which we take the average grade.

    If there are any huge differences in scoring the exam between the different experts, this can be flagged up to take a closer look at.

    – Based on the number of volunteers we would set a maximum on the amount of people that can go through the certification process each month. If we were to set up this project, I was thinking of initially having around 15 people/month.

    These are just some additional thoughts.

Comments are closed.

%d bloggers like this: