Thursday, April 24, 2008

Proxy test takers, item harvesters and cheaters... be very afraid

"I've been in testing since the 1980's with Control Data's first computer testing implementation," said Randy Trask, Vice President of Market Development for Pearson VUE, "and I can tell you that cheating has always been around. What's changed is the pervasiveness of the internet. In a half second, an individual can compromise intellectual property around the globe in forums that are highly categorized and easily accessible."

That's true.

Trask said that VUE and Cisco had identified three main categories of people who attempt to compromise certification exams: proxy test takers, item harvesters and cheaters. "When we began to devise counter measures, we developed techniques directed to each of the three groups," said Trask.

It's interesting that Cisco hasn't included another category that causes the compromising of its certification exams : its own people who create the kinds of questions that lead people to be "cheaters". And if there weren't cheaters, there wouldn't be any need for item harvesters too.

Proxy test takers go to a test center and sit for an exam registered as another individual. "It's an age old problem," said Trask, "but it's definitely a problem that we continue to see."

That's a real problem and everyone pretending to be another one should be prosecuted.

An item harvester is someone who takes an exam under their real name, but their sole intention is to steal intellectual property. The worst offenders in this category are those in the employ of black market test vendors or other organizations who believe they can profit from stealing test questions. A less sinister item harvester is someone who used a brain dump site to prepare for the exam and returns to the site to post several items that he remembers to "give back" to the group.

These are the people that are usually being paid to do such a "job". If there wasn't a respective need by candidates, these people would be far less. The need for something usually drives the market.

Cheaters are individuals who are not trying to profit stealing intellectual property; they are just dishonest people willing to cheat to pass an exam. "It's like when you were in school and somebody wrote the answers on the bottom of his shoe," said Trask.

I suppose most of them would do the same even if the exams were changed, in order to avoid the usual maxim "there is the correct answer, there is the wrong answer, and there is Cisco's answer."

It's like the people who continue to download pirated movies/songs even if some of them are costing less than their online time. They are addicted to this kind of activity.

But i believe that there is a considerable percentage of people who become cheaters because Cisco is pushing them to that way. This is surely not a good excuse but lost time and money are the two main factors for driving them in that direction.


Candidate Authentication Program

"In a proxy testing investigation, being able to review photographs associated with exams is a powerful investigative tool," said Trask. "If we have the same photograph appearing under different names and signatures, it provides evidence that is difficult to refute."


Let's hope that we don't end up having people being masqueraded in the exam room.

Exam Data Forensics

During a Cisco certification exam, each keystroke is logged and a record is created that includes the length of the test period, how much time was spent on each question, whether an answer was changed, how much time was spent on the second answer, etc. After the exam is completed, but before the results are processed, each exam session is analyzed by forensic software that analyzes the session against established behaviors and suspect exams are flagged for investigation.


That is an interesting approach. Usually you can't answer a question in 15", unless you already know the answer, because you have memorized it. But now, all cheaters will delay their answers and will try to act like they're thinking. Maybe a neuro-electrical head-device translator would solve this problem too?

Program Data Forensics

Another layer of security examines a wide range of other program attributes to determine what is occurring within the program. "We're comparing testing center to testing center, and within a center, we are comparing one administrator to another administrator to see if inconsistencies emerge," Trask said. "We look at things like candidates who live in one country but test in another; we review financial information, credit card information; just a very wide range of information to see patterns and inconsistencies."


Are there any candidates who travel to another country just to give a written exam?

Consequences

In the past when someone was caught cheating they would have their score invalidated and their credential could be revoked. The most serious consequence would be preventing the individual from participating in a certification program. But the stakes are going up.

"Our evidence collection is now done for evidentiary purposes," said Trask. "We intend to pursue civil and criminal remedies against people who choose to profit from violating or infringing on our intellectual property. And I applaud Cisco for it. Previously people wanted to avoid any bad PR related to going after these groups, but both Pearson and Cisco are taking it very seriously and we will take action against them."


Are there going to be civil and criminal remedies for cheaters too? Can the forensics stand up as a proof in court?
Anyhow, i guess now people will -actually- think twice before cheating...


Awareness and Engagement

Cisco and Pearson are developing targeted messaging that will be delivered in email campaigns to certification holders and on posters in testing centers. One central theme will be appropriate vs. inappropriate study methods and behavior. "In many instances, candidates don't see anything wrong about using or contributing to brain dump sites," said Trask. "We are going to be educating people about the local and federal laws governing theft of intellectual property in the various regions around the world."

Another effort will be to engage certification holders to alert program representatives in cases where people don't seem to have the requisite skills for a certification. "People who have earned a certification honestly have a vested interest in protecting the integrity of the process and the value of their certification," said Trask. "Anytime someone has a certification but doesn't have the skills to do the job, it hurts the value of everyone's certification."


I still don't see any plans for changing the exam type. For adding questions/labs that make actually the candidate think and not memorize a book or a dump.
How about mixing the order of answers? That should be an easy one.
How about adding new answers and removing old ones from the same question? This should be easy too.
How about removing all those needless questions that are put on purpose there in order to confuse you?
How about taking into account the candidate's comments and removing all the questions that receive a massive negative reception? Then these candidates should be given a free second chance (if they failed for giving wrong answers to these questions).
How about using dynamips and providing more realistic simulation labs instead of the silly "90% of commands not supported" ones?
How about giving a free recertification every 2 recertifications?
How about giving free lifetime recertification (twice a year) for CCIEs that have done exceptionally well in the lab?

You can read the whole article here.

5 comments:

  1. I find it interesting that you say that Cisco is pushing people to cheat. What makes you say that?

    I personally believe that Cisco could take some relatively simple steps (some of which you mentioned) to reduce the effectiveness of the most common cheating methods. But so far in my Cisco journey (which has only covered the CCNA and my BSCI tests) I have not had the need to cheat.

    I think people's real or perceived pressures lead them to cheat. For some reason, just having the certification title becomes more important than building the skills and knowledge to really earn the title. People justify cheating to try and convince themselves and other's why it was OK for them to break the rules. But in my book, anyone who needs to cheat to pass a certification exam is just a poser.

    ReplyDelete
  2. If you could have a proctor at every written exam where you could ask explanations for questions that you don't understand their meaning, it would make cisco understand why many people prefer to memorize the answers to such kind of questions. These questions are the ones that should be removed as soon as possible.

    Of course, after solving the above problem, people will continue to cheat.

    I agree with you that the certification title is a major reason by itself to make the people go cheating. They don't care about learning, they care about "status". That's why many employers are actually retesting certified people and find out how miserably they fail when cheating is not allowed.

    It all depends on your motivation. Do you want to show better or be better?

    ReplyDelete
  3. I've been able to do well on Cisco certifications and tests without cheating, but I've lost a lot of points due to unclear questions where more than one answer could theoretically be correct. This isn't a big deal for me, as these kinds of questions usually lower my grades from 90%-100% to 85%-95%, but they could pose a problem for people who barely pass the courses and certification exams.

    As for the cheaters, anyone who has worked in the field will tell you that experience means a lot more than a certification. Once you're getting paid a good salary to do a professional job, you'd better know your stuff, or it'll catch up to you eventually. It's a matter of learning before the exams or learning when you have no other choice on the job. Surveillance is extremely slack in my classes, and I've seen people Google answers or open text files containing notes right in front of the professors, who were busy playing Solitaire or something.

    ReplyDelete
  4. The problem with Cisco is that they too have become "cheaters" in the fact that they in multiple choice answers give two answers which are very simular, thus giving rise to the argument that they are cashing in on the certification.

    I have been taking re-certifying since 1998 and i can tell you that the exams have changed not for the better.

    ReplyDelete
  5. I think reducing the test centers and employing direct cisco /vue employees for exam proctoring would help to a certain extend.. Tricky questions from cisco are definitely a good way to fail candidates and there by ripping them off their hard earned money..
    But again, who are we kidding? Cisco needs the money, whether the exam is done by a proxy test taker or not.. I personally have reported about three different institutions couple of months before - about soliciting proxy exams. So far no action has been taken against the said exam centers.

    ReplyDelete

 
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Greece License.