More on the BSD Professional Exam

BSDCan, the premier BSD conference held annually in Canada hosted a talk on the BSD Professional (BSDP) exam  given by yours truly at this years conference.  The talk discussed how the BSD Certification Group is currently creating the BSDP exam including the written form and a "hands-on lab".  If you couldn't make it to Ottawa this year you can still hear the talk and see the slides.  Many of the talks were videotaped.  The BSDP talk can be found at
http://www.fosslc.org/drupal/content/consideration-bsd-professional-exam.

Of particular interest are the lab requirements- 11 in all- that will form the basis for selecting the eventual lab design from the many ideas described throughout the talks.  If you are only interested in the discussion about the lab itself, you can fast forward to about 35:50.  That section presents an example lab and describes the types of things that the candidate would have to perform in the hands-on portion of the exam.

The actual lab tasks are drawn from today's news- a disaster has hit Example Corporation, and your boss Mr. Big, wants you to get things working again, using new 3 new BSD systems.

There are specific requirements for filesystem layouts, network connectivity, and DNS. A description of an example lab virtual environment is also shown.

Be sure to check out the live demonstration after the talk.  The demonstration, though different from what is described in the talk, shows how four virtual machines can be set up in a high  availability firewall environment, using two OpenBSD guest virtual machines, a DragonFly guest system, and a FreeBSD guest system. In the demo, the DFly and FreeBSD hosts  (external and internal hosts respectively) exchange data seamlessly through the  OpenBSD CARP and pfsync interfaces, just as they would in a real world physical environment.

Even more news about the BSD Professional Exam is coming up, including how you can contribute.  Stay tuned!

BSDP Job Task Analysis Survey Results

Development of the BSD Professional Certification Exam is officially underway. A Job Task Analysis (JTA) survey for the exam was held in October and November of 2009. The results have been collated and the report titled "BSD Professional Certification Job Task Analysis Survey Results" is now available at http://www.bsdcertification.org/downloads/BSDP_JTA_Report.pdf.

The 93 page report details how the BSD community evaluated the exam knowledge domains and specific items in each domain. Each item was rated for its "Importance" and "Frequency" for a typical senior level BSD system administrator. The results will help determine the items that will be a part of the BSDP exam.

As with the previous JTA for the BSDA exam, the comments make a very interesting part of the survey report. Some were very vocal about specific topics such as installation and security. It was actually refreshing to see how much interest there is in the BSD Professional exam. Check it out for yourself!

Blooper Reel

Just like those movie reels where you see funny mistakes or gags that occur when the actors mess things up in filming the movie, the BSDCG got to see a blooper of our own recently.

After receiving his exam results, one of the examinees noticed that the scores listed on the exam results PDF file seemed a bit odd. He contacted us through the website and mentioned that he thought the results were incorrect. Turns out- he was right.


The figure above is from the incorrect exam results PDF that was sent out to our sharp eyed exam candidate. The figure shows, from left to right, the Knowledge Domains on the exam, the candidate score for each Knowledge Domain, and the Group Minimum, Average, and Maximum sub scores for each Domain. The "Group" scores represent the scores for all recent attendees for the exam. It shows the Group Minimum score, and the Average and Maximum scores.

Notice anything unusual? Check the Group Maximum score for "Basic System Administration". It's listed at "17%". Now ask yourself- "Is it really true that the highest score for Basic System Administration could really be just 17%??" especially for an exam that is over one year old?

Turns out- it wasn't. Below are the corrected scores for the same candidate.

As you can see, the Group scores (Minimum, Average, and Maximum) appear closer to what you would expect in the real world. That "17%" for a Group Maximum regarding "Basic System Administration" is really "92%"- reflecting the fact that the group scores are significantly better than what was previously posted.

So.... what went wrong??


Like many system failures, this one was more than just a single mistake. One factor was that we've recently instituted a new exam "form" (separate set of test questions) and our exam scorer decided to roll the results into a new file. However, it's not just one file that has to get updated- there are several files and we missed one.

At the same time, our reviewer who mails out the exam PDFs (uh, that's me) didn't carefully review this set. Actually, I did notice that the scores were strange, but assumed it was the result of the form change.

Once we were alerted to the issue, we rechecked and corrected the procedures we use and reran the exam scoring for everyone who took that set of exams. It turns out that the Pass/Fail status for the exams were not affected- just the individual and Group scores.

For us, this highlights again how important community involvement and feedback is to the success of the exam. In this case, we found the mistake and corrected it immediately. And I'm sure glad we did. It brings the importance of reviews back into focus. You can bet we will all be reviewing all exams in the future much more carefully!

New Video- "BSD Certification Group: A Case Study in Open Source Certification"

Dru Lavigne, Chair of the BSD Certification Group, gives an informative talk about the creation of the Open Source certification system for BSD operating systems. This talk covers the BSD Associate (BSDA) and BSD Professional (BSDP) certifications and describes how they were developed. It presents the value of psychometrics and the process of creating a psychometrically valid exam including

• Job Task Analysis - Determining which skills are frequently used and how important they are
  
• Exam requirements - Determining what should be covered on the exam
  
• SME Question Development - Creating and validating test questions
  
• Beta Period - Creating and distributing a limited version of the new exam
  
• Angoff Session - Determining the final pass/fail score for the exam
  
• Final Launch- Getting the exam out the door
  

The talk also discusses other goals of creating Open Source exams- affordability, local availability, the encouragement of local study groups, commercial training materials, Open Source Testing Centers, and more. Finally, it shows many ways you can help improve BSD Certification and Open Source exams.

The video takes about 35 minutes and is available here, or you can just click on the image below.

Thanks to Patrick McEvoy and the fine folks at Manhattan Neighborhood Network for putting this professional video together, and the Internet Archive website for hosting.

Enjoy!

BSD Professional Job Task Analysis Survey

The first step in getting the BSD Professional exam underway is the Job Task Analysis (JTA) Survey. What is a JTA? It's an online survey where you can help define what skills are important for a BSD systems administrator at the "Professional" level.

The survey presents a collection of tasks organized into Knowledge Domains. Each task includes two rankings- one for Importance and the other for Frequency. These rankings are used in Psychometric Analysis to identify the key skills that should be tested on the exam.

The survey takes about 20 minutes to complete and contains several opportunities for you to express your thoughts on the selection or description of tasks for each Knowledge Domain. Your comments are very important, as they help form the direction and style of the exam.

The survey will be provided in other languages besides English as soon as our translation teams get finished. We hope to have the survey available for 30 days in each language.

As soon as the survey is complete, we'll start compiling the results.

One of the questions asked is about whether the BSD Professional exam should be just a multiple choice exam, just a hands-on lab, or both. We are very interested in your opinion on this issue. There are those who say certifications are useless. But if you can really assess skills, like you can with a hands-on lab exam, this makes the certification verifiable, and provides real value to potential employers.

Let us know what you think. Give the BSDP JTA Survey a go and express yourself.

Update on the BSDA Exam

The BSD Associate Exam is now over a year old! Here are some interesting stats so far:

• 12 Events in all of 2008; 14 events in just the first half of 2009
• Over 1000 people have registered for a BSDCG ID (needed to register for an exam)
• The exam has been held in US, Brazil, Canada, Germany, Japan, France, Denmark, Ukraine, Netherlands, Argentina, and the UK
• So far, 66 people have passed the BSDA exam and received their certificates
  

As of April, 2009, there are 6 exam locations coming up: Canada, Germany, Argentina, UK, Netherlands.

If you know of an upcoming conference near you, let us know (send email to www AT bsdcertification.org) and we'll get in contact with the conference organizers to see if they will provide a time and place for the BSDA exam. We'll bring the pencils, you bring the know-how.

In Other News...

We're in the process of reviewing the exam for the next version. Our psychometrician, Dr. Sandra Dolan, has provided us with a report of the exam results for the first year. BSD subject matter experts, drawn from the BSD community have helped modify some the the questions to increase the accuracy of the exam.

The Winter 2008 DVD is out. This edition includes the QEMU virtualization software (and the AQEMU front end) that enables working with multiple BSD systems at the same time. Additional information on using the software is here [bsdwiki.reedmedia.net]. Purchasing a DVD is an excellent way to prepare for the exam, helps fund the exam development effort, and helps keep the cost of the exam globally affordable (still just $75USD).

Planning for the BSD Professional Exam (BSDP) is underway. We will start with a Job Task Analysis (JTA) survey similar to the JTA we produced for the BSDA exam. The BSDP JTA should be ready in a month or two, and we will announce the survey through all our regular media channels. Stay tuned for the next level.

BSD Associate Certification Adv. in BSD Magazine

If you are in your local Barnes and Noble or Borders book store, you might see a copy of a new magazine devoted to all things BSD- BSD Magazine.

Published by the same folks who put out Hakin9, an excellent magazine devoted to information security, the new BSD Magazine has several cutting edge articles about BSD. Some articles in the current issue include:

• Pushing BSD as an open source desktop...
• PC-BSD overview...
• Sguil 0.7.0 on FreeBSD 7.0...
• How to Dual-Boot Vista with BSD – a step-by-step approach...
• Keep smiling, waste spammers' time...
• OpenBSD pf – the firewall on fire...
• Instant Messaging with Jabber/XMPP...

and more. The issue includes a DVD containing FreeBSD 7.0, documentation, and some extra applications.


And if you happen to look closely, you'll see the very first advertisement for the BSD Certification Group. The ad is for the recently released BSD Associate certification.

Special thanks go to the fine folks at iXsystems who helped developed the professional quality graphics and layout. That's their logo in the lower right corner.

We hope to have more advertisements in BSD Magazine in the future.


The BSDA certification is gathering steam. We've presented the exam at conferences in Los Angeles, Ottawa, Krakow, Brussels, Toronto, Berlin, Ede, and Chemnitz. So far, over 400 people have registered for a BSDCG ID, which you need to do before you register for an exam. Some of the comments we've received from the exam said it was "harder than I thought it would be" and it "made you think". Be sure to read the Exam Objectives, currently available in English, Mexican Spanish, and Russian.

If you would like to see the exam at a conference near you contact us, and we'll try to arrange for an exam at that conference.

Certificates, Seals, and More BSDA News

The certificates are back from the printer, and they look good! They have a blue border with dark blue and black printing on a heavy, light grey card stock. The seals are embossed gold foil and are applied separately. The seals look very sharp. The actual certificate doesn't have the dark brown border, but it is positioned so you can use a nice frame for your wall or desk. The whole thing looks very attractive.

We've held the BSD Associate Exam at three conferences so far- SCALE, FOSDEM, and Linux-Tage Chemnitzer with over a dozen candidates. All the exams haven't been scored yet, but they should be finished in a few days. Those sharp enough to pass will get one of these certificates as soon as we can send them out.

The next conferences to host the BSDA exam are Flourish (Chicago, IL, April 4-5, 2008), IT360 (Toronto, Ontario, Apr 09, 2008), NLUUG (Ede, The Netherlands, May 15, 2008), and CONFidence (Krakow, Poland, May 15, 2008).

See the list of upcoming conferences, and register for a BSD Associate exam at https://register.bsdcertification.org//register/events. Cost of the exam is $75 USD (or half that if you took the beta exam). More conferences are being added all the time, so check back frequently to see if there is a conference near you. If not- let us know! Select the Info subject through the "Contact Us" page at http://submit.bsdcertification.org/contact.

Remember, you have to obtain a BSDCG ID before you register for an exam. The BSDCG ID is free, and we keep your ID information private. It is not shared with anyone.

As always, more information on the BSD Certification Group and the BSDA exam is found at www.bsdcertification.org.

BSDA Exam Launched!

The BSD Certification Group launched the first BSD Associate Exam at the Southern California Linux Expo (SCALE) conference held in Los Angeles, Ca. , last weekend. As it was, there was not enough time to promote the first exam, so the number of candidates was limited.

The next BSDA Exam will be held at FOSDEM in Brussels, Belgium, on Saturday, Feb 23, and Sunday, Feb 24. Seating is limited, and you must pre-register. We already have over 30 registrations for FOSDEM. Also coming up is Linux-Tage Chemnitzer(link in German) in Chemnitz, Germany in March, 2008. All exams are paper based and are in English. Cost is only $75 (or half that if you participated in the beta exam.)


So, we're underway with the first exam! Sign up for a BSDCG ID and get to a conference near you to take the BSD Associate Exam.

Get Involved. Get Certified. Get Ahead.

BSDA Certification Moves Forward

Last weekend, members of the BSD Certification Group met in New York to finalize some standards for the BSD Associate exam. The exam is now in its final form- all that is needed is to select the actual 100 questions to be used in the first version. Our psychometrician, Dr. Sandra Dolan, is pleased with the progress so far. We will have a psychometricaly sound exam, that will provide a solid basis for determining the skill level of all candidates.

And the latest news: we now have two slots for the exam at SCALE- the Southern California Linux Expo. A big "Thank You!" to the folks at SCALE for allowing us to hold the exam during their conference. This is our inaugural exam, and we are looking forward to it!

Several other conferences have generously allowed us to hold exams. See the list, and register for an exam at https://register.bsdcertification.org//register/get-a-bsdcg-id

Thanks to all our supporters and friends for getting us to this point!

News on Credit Cards, the Beta Exam, and More

We've cleared another hurdle- we're now accepting major credit cards for payment on the website. If you've been wanting to purchase the Summer 2007 Courseware DVD with a credit card instead of PayPal, you can now do so.

There are also two more brochures on the Resources Page- one in German, and the other in English. These are attractive tri-fold giveaways, suitable for use at conferences, BSD User Groups and anywhere a bunch of BSD folk hang out.


We're slowly getting the first certification ready. Psychometric analysis is underway, and we expect to have the first exam ready in the first quarter of 2008. There is still a long way to go. Subscribe to the BSDCert email list and send in your thoughts and ideas.

We're Back!

Although you might think this blog is dead, it's actually just been in hibernation :-) Not so the certification effort itself.

A lot has happened since the last post on Feb 2006. If you've been following news on the BSD Certification Group website, you'll know most of these news items. We've:

• Held a logo design competition (see the new design on the BSDCG home page)
• Held a fund raising drive among BSD user groups, that helped us raise money for our new psychometrician
  
• Selected a psychometrician, Dr. Sandra Dolan, to help develop the exams
• Completed the Test Delivery Survey, showing the number and location of interested test takers
• Completed the first round of psychometric analysis with a beta version of the BSDA (BSD Associate) exam. Almost 100 individuals participated in the exam on three continents. The results will help improve the final BSDA exam coming soon.
  

And those are just the highlights. Additional news on all of these items and more will be forthcoming in the next several weeks. Good things are (still) happening!

News Bits from the BSDCG

BSD Certification Group Acting Chair Dru Lavigne will be speaking on BSD Certification at the UKUUG Spring Conference in March 2006. Dru will bring the audience up to date on our activities, and highlight plans for this year. Hope you can join the fun.

Currently, we're looking at the best way to publish a study guide. If you've ever written a book, you know how daunting it is to navigate the content, schedule and the publisher all at the same time. We have a good grasp on the content, and right now, the schedule is open. It's the publishing piece we're trying to nail down. Send us a note on the mailing list if you have any suggestions for effective publishing.

Another research topic right now is in the area of OSS test delivery and administration systems. A few solutions exist, though none really meet our requirements. One that comes close is the MARTest system, developed by the Zope Corporation. Other suggestions are welcome.

Our application for 501(c)(3) status with the IRS was sent several weeks ago. No word yet, but we are hopeful. We'll post an announcement when we get the word.

Look for an announcement about our BSDCG elections soon too. Details are not yet finalized, but we should have an announcement this month about our first meeting, and our election of officers.

Happy New Year!

It's a new year- and I'll bet I'm the last one to wish it to you...

Wow- it's amazing how fast the weeks fly by. This is the first post since November! In addition to the holiday stuff, I've moved my own apartment and relocated to a different city for a project. So my apologies for not getting back to you all sooner.

Time for a quick update...

We're actually receiving donations now. Our donation button is active and we're receiving money. Please donate! Our success in producing the first certification is directly tied to your support. Thanks!

We're also investigating Open Source test delivery mechanisms. If you know of a good delivery system let us know. Our goal is to have the system handle as much of the test question delivery, scoring and statistics as possible. Give us a shout if you know of a robust, secure system.

There are also some 'research groups' springing up on topics like researching what companies use BSD, contacts, and other assorted topics. If you have some time and want to contribute to these research projects send a note to the BSDCert email list. (Subscribe here.)

We're looking forward to a lot of work and progress in 2006. Stay tuned for more news.

Tracking Donations

The BSD Certification Group Donations Button is now officially open for business! Please join us in supporting BSD Certification. We're hard at work on the certification standard and we need your help to continue making it possible. If you would like to be listed on the donations page, let us know. Or if you would prefer to remain anonymous, we can do that too. Contact donations@bsdcertification.org for more details.

BTW- we're thinking of adopting a donations thermometer to track our progress and give visibility to the generosity of all our sponsors and contributors. Here's a sample (not the real one yet) of what it looks like. The script is written in perl and uses GD::Simple for the graphics stuff. It was fun to write. Let us know at info@bsdcertification.org if you think it's a good idea, or would like a copy of the script.

Donations, Sponsors and Plans- Oh My!

The BSD Certification Group website will be modified in the coming days to add a donations button through PayPal. We're in the final stages of testing with four major credit cards as well as other PayPal accounts. Hopefully, this will enable everyone to donate to our project. And right now, donations are sorely needed! We're trying to engage a company for psychometric analysis for our exams, and the projected cost is over $30,000. So, when the donation button is ready- please be generous.

We are also continuing to look for sponsors. If you would like to help sponsor this effort, or know someone who might be interested in being a sponsor, please send an email to info@bsdcertificationgroup.org.

In other news, I'm sorry I haven't been able to keep the project plan updated recently. My DSL went out at about the same time my wireless card broke. As a result, I'm without Internet on my BSD box that has the plan. As soon as this is fixed, I'll update the plan.

Speaking of plans, the business plan is ready. If you would like a copy of the plan, please drop a note to info@bsdcertificationgroup.org. We're going to send it out and follow up with those who request it.

BTW- there have been several interesting articles about BSD systems in the news lately. Good all around.

BSD Usage Survey Released!

Well, the Usage Survey is out the door. That's good news! There aren't very many sources of BSD usage information- and we hope this fills that gap. Maybe we can make it an annual thing. We'll see...

The survey is pretty comprehensive as far as determining how BSD systems are used and who uses them. But, as you'll read in the report, there are ways we can make it better. For example, just pre-screening who receives and processes the survey makes it much easier to determine if there are duplicates from the same company.

So- read and enjoy; especially the comments. Again- very good responses from all over the world. BSD truly is international!

Survey Report, Web Site, and Incorporation News

The Usage Survey Report is coming along. A bit slower than the last couple of documents, as we've all been busier than usual with $REAL_JOB. Hope to get more done this weekend on the Report.

The new website is up now, and everything works. We had some problems with the contact form, but it should be working now.

Incorporation is complete! We got our offical papers from the State of NJ. We can now open a business account and get ready for the flood of donations. You are going to donate aren't you? Hope so... :-)

BTW- the business plan is ready for publication. Use the Contact Us form on the website if you want a copy. Or just post to the mailing list. We'll get started on a wider distribution this soon.

BSDA Certification Requirements Document Completed!

After some last minute scrambling, we finally got the BSD Certification Requirements Document done. The document lists seven different study domains and has over 100 different topics. Here's the PDF copy. And a humorous note- even after all that checking, the first version we put up had an 'ipconfig(8)' in one of the entries! Thanks to Hugo Silva, one of our translators, for pointing it out.

Also- the Business Plan is now available. We're releasing it to our Advisory Board first, then to the general public. If you want an advance copy- send an email to info AT bsdcertification dot org.

We're now looking for sponsors. If you or your company has benefited from BSD, and would like to sponsor development of the certification, please contact us. We need your help!

Next up is the report on the BSD Usage Survey. We're drafting that right now. And just like the first survey (Task Analysis Survey), this survey is loaded with comments. There are hundreds of comments. Here are a few samples:

• We're a small start-up. The BSD family of OSes has enabled us to create products and provide an on-line store using no-cost systems. Without BSD we would never have got off the ground. A big thank-you to everyone who has contributed in whatever way. We've been supporting the cause for several years now and hope to be able to increase funding as our business grows.


• BSD, in any form, is an excellent product and a very robust OS. I am forced to use Windows for one program, but, everything else is done in BSD.


• And my favorite: Hurry up :-)

Rest assured, we are working as fast as we can!


Side note- Unfortunately, I've had to turn on word verification to prevent spam commenting. My apologies, but it will save me quite a bit of time to remove them. But please don't let that prevent you from commenting :-)

A Hectic Two Weeks

It's been a hectic two weeks. There is a lot going on right now.

We've just finished up the BSD Usage Survey. The data needs some massaging before we are ready to publish a survey report. That will be a few days yet. Over 4000 people responded to that survey. Wow! I was impressed when 500 responded. The survey was very popular, and there were many comments. We'll devote space to the comments again, like we did for the Task Analysis Survey Report. The comments are really refreshing to read.

The BSD Associate certification requirements document is nearing completion. This has been a huge effort. Many people contributed ideas and suggestions, and it's up over 50 pages now. The document contains seven "study domains" that contain about a dozen or so exam objectives each. It's quite detailed. One nifty feature is in the appendix- a chart listing the four BSD projects and common administrative commands and files among them. This should be quite useful to BSD students, young and old.

A lot of time has gone into getting the Business Plan shaped up. The plan is done, and is now being reviewed by the Group. It contains an operating plan for the next two years, and shows three revenue streams- sponsorships, donations, and exam fees.

Speaking of business, that's where more time went- into researching and drafting articles of incorporation and bylaws. We've decided to incorporate as a non-profit organization and we plan on submitting an application to the IRS as a 501(c)(3) scientific and educational charitable organization. If the IRS accepts our application, contributions may be tax deductible. That's good news for us, and for donors too! More details as they become available.

And if all that wasn't enough, I'm transitioning into a new role at my company. More responsibility, and (probably) more work to do.

And that starts in just a few hours- so I better get some sleep!