Machine generated captions for BSD conference videos

One of the most frequent requests I've received, since Launching the BSD Conferences YouTube channel last year, has been for captions in Spanish, Russian, Chinese, and other languages. I was excited last month when Google announced automatic captions for Youtube videos using machine translation. This feature is still highly experimental but I am happy to report that it has been enabled for the BSD Conferences channel. In combination with the much more mature automatic translation feature, this means that captions are now available in over 50 languages from Afrikaans to Vietnamese for most of the 73 videos in the BSD Conferences channel.

The automatic captions are still highly experimental and the quality of transcription for highly technical content spoken by a diverse set of international speakers is a significant challenge to get right. If you are interested in helping to correct any of the English transcripts I would be happy to provide you a simple text file of the transcription, with each line offering the start and end time for the caption to be displayed, and the caption text. One advantage of the machine translation is that the most time consuming part of manually creating captions, synchronizing the timing of the text with the speech, has been done automatically. Even when the technical words are mangled, the timing information in the automatic captions files can be leveraged to make the process of manually improving the captioning much easier.

The experimental automatic captions are only available directly from the video watch pages, and not from channel pages or other views. For example, visit www.youtube.com/watch?v=nwbqBdghh6E to see one of our most popular videos, Kirk McKusick speaking on FreeBSD Kernel Internals. Hover over the triangle at the bottom right of the video, then over the CC submenu and select "Transcribe Audio". You can then choose to "Translate Captions" into a different language as well.

The Slashdot Effect

After 8 months, 66 videos uploaded, and 141,676 views, the BSD Conferences YouTube Channel was slashdotted for the first time last week. Specifically, Theo's OpenBSD Release Engineering talk was linked from this slashdot post. Views of the video spiked to nearly 8,000 a day after the Slashdot post, which dwarfs the previous highs of around 1,500 videos a day after I posted about Kirk McKusick's FreeBSD Kernel Internals lecture.

I think this is an excellent reminder of the power that forums like Slashdot still have in directing traffic among those seeking technical content online. I would encourage anyone interested in seeing more BSD related content online to install browser bookmarklets, toolbars, or other shortcuts to more easily share and promote FreeBSD content on Digg, Del.icio.us, Slashdot, etc..

FreeBSD Code Metrics Now on Ohloh.net

I've written previously about Ohloh.net and how I'd like to see more of the dynamic code metrics calculated there available on the FreeBSD web site. I am happy to report that today I noticed after several years of attempts, the ohloh repository import servers have finally managed to get through the entire FreeBSD source repository. Their software setup previously had difficulties dealing with a project with as long of a history as FreeBSD.

You can now view the top level code metrics about FreeBSD from the FreeBSD Project Page on Ohloh.net. This page indicates that there are over 10 million lines of code, that more files are licensed under GPLv2 than any other license.

The committer totals do not quite match up with Peter's Commit Counters. Even after accounting for the fact that Peter's system could potentially double count a commit that touches both sys and non-sys parts of the source tree, the numbers from Ohloh are still lower for some committers. Unlike the numbers on cia.vc, the FreeBSD project on ohloh.net only contains the source repository. We are currently lacking the anonymous cvs access to our doc repository necessary to add the doc project to ohloh.net.

How do the numbers reported on ohloh.net compare to code metrics others have reported for FreeBSD? Does this match expectations or are there any major problems with this data? How can we use this information on our website? Would a badge on the front page showing "Last improvement made X minutes ago" be useful? A list of most active committers in the past week on one of the developer pages? Other ideas for utilizing the work the Ohloh project is doing?

Open Source in Recessions

In general, recessions can be really good for open source. Large businesses look to cut back on IT budgets and this often involves re-evaluating whether proprietary software and maintenance contracts are necessary, given high quality open source alternatives. Companies may dedicate more internal resources to open source projects, and also the surplus of underemployed engineering talent in the market may be available for more open source development work.

Unfortunately, there are also some significant downsides, and one that I would like to highlight is the plight of the small open source shops around the world. We've seen pleas earlier this month to Save BSD Magazine, and in recent years some other smaller open source companies such as Daemonnews, and the Japanese publications FreeBSD Press and BSD Magazine have exited the business. In the current environment I would like to take the unusual step of plugging a company that has been selling, marketing, legally defending, and supporting FreeBSD from the very beginning.

FreeBSD Mall has been selling FreeBSD CDs since 1.0 in 1993 and is still selling and supporting CDs, DVDs, books, branded apparel, and more. The PC-BSD live-dvds make an excellent introduction to FreeBSD for new users and the complete FreeBSD DVD distributions are quite handy to have. Consider spending a few dollars at the FreeBSD Mall site, buy a BSD Magazine subscription, or otherwise spend some money to encourage the small commercial FreeBSD ecosystem and perhaps contribute to more funds being available to exploit the many disruptive opportunities (netbooks, cloud computing, etc..) that could be very good for open source and FreeBSD during this recession.

Remaining AsiaBSDCon 2009 Videos Posted

The remaining 9 videos from AsiaBSDCon 2009 have been posted. The new videos include talks by Theo de Raadt, Eric Allman, Kris Moore, Mohamad Fauzie, Brooks Davis, Atillio Rao, A. Kantee, and the Works In Progress Sessions.

• AsiaBSDCon 2009:The OpenBSD Release Process: A Success Story, Theo de Raadt
  
• AsiaBSDCon 2009: Internet Mail — Past, Present, and (a bit of) the Future, Eric Allman
  
• Kris Moore: PC-BSD - Making FreeBSD on the Desktop a reality
  
• Mohamad Dikshie Fauzie: FreeBSD and SOI-Asia Project
  
• Isolating Cluster Users for Performance and Predictability, Brooks Davis
  
• A. Rao: The Locking Infrastructure in the FreeBSD kernel #2
  
• A. Rao: The Locking Infrastructure in the FreeBSD kernel #1
  
• A. Kantee: Environmental Independence: BSD Kernel TCP/IP
  
• Work-in-Progress Session in AsiaBSDCon 2009
  

Thanks again to Hiroki Sato for posting the videos and organizing 3 consecutive years of successful AsiaBSDCons. Sato-san has also created two separate YouTube playlists for the AsiaBSDCon 2008 and AsiaBSDCon 2009 videos. These playlists make it easier to find the newest videos from among the 66 videos now in the BSDConferences YouTube channel.

First AsiaBSDCon 2009 Videos Posted

Thanks to Hidetoshi Shimokawa and Hiroki Sato, we now have 7 videos from the recent AsiaBSDCon 2009 in Tokyo, with more on the way:

• A. Zakharchenko, Mail system for distributed network


• K. Dzonsons, Deprecating groff for BSD manual display


• D. Gwynne, Active-Active Firewall Cluster Support in OpenBSD


• R. Jaworowski, FreeBSD on hi-perf. multi-core embedded PPC


• M. Warner Losh, An Overview of FreeBSD/mips, AsiaBSDCon2009


• Constantine A. Murenin, OpenBSD Hardware Sensors Framework


• Philip Paeps, Crypto Acceleration on FreeBSD, AsiaBSDCon2009

This makes 57 videos in the BSDConferences YouTube channel. We now have over 1,000 subscribers to the channel. Kudos to both the DCBSDCon and AsiaBSDCon organizers for getting the videos processed and uploaded shortly after the respective conferences when interest is highest. The videos from AsiaBSDCon 2008 were also made available earlier this year.

Nominations for Google-O'Reilly Open Source Awards

Nominations for the 5th Annual Google-O'Reilly Open Source Awards are now being accepted. As in previous years the awards are open ended so nominations should also include the name of the award / achievement in addition to a nominated recipient (e.g. Best Lifetime Contributor, Community Builder, Tool Builder, Overall Hacker, etc..). Nominations are due by May 22, 2009 and will be judged by a committee before winners are presented with an award at OSCon 2009. (Via Google's Open Source Blog.)

50th BSD Video Posted: All DCBSDCon '09 Videos Live

Jason Dixon has made available the last 4 videos from DCBSDCon 2009. The last video marks the 50th video uploaded to the BSDConferences YouTube channel. This channel was created less than 5 months ago and now has 924 subscribers from authenticated YouTube users, and the videos have been viewed over 76,000 times by users from around the world (includes partial views).

The newest 4 videos are :

• Richard Bejtlich, Network Security Monitoring using FreeBSD
  
• Henning Brauer, Faster Packets: Performance Tuning in the OpenBSD Network Stack and pf.
  
• Jason Dixon, BSD is Still Dying (closing remarks)
  
• Chris Buechler, Network Perimeter Redundancy with pfsense
  

The average number of daily views is around 500, with significant spikes above 1,500 in the days after a popular new video is announced:
And the top 10 videos sorted by views (biased towards older videos that have been available longer) are :

• FreeBSD Kernel Internals, Dr. Marshall Kirk McKusick
  
• PC-BSD: FreeBSD on the Desktop
  
• New features in FreeBSD 7
  
• BSD is Dying, Jason Dixon, NYCBSDCon 2007
  
• ZFS in FreeBSD, by Pawel Jakub Dawidek
  
• FreeBSD, Building a Computing Cluster
  
• FreeBSD Profiling, Kris Kennaway, MeetBSD 2008
  
• Embedding FreeBSD, MeetBSD 2008
  
• BSD v. GPL, Jason Dixon, NYCBSDCon
  
• FreeBSD Network Stack Optimizations, Robert Watson
  

20 FreeBSD Summer of Code Students Announced

Leslie Hawthorne from Google today announced the students selected to participate in this year's Google Summer of Code. Among the 1,000 students from 70 countries participating in this program, 20 will be working on FreeBSD projects, 11 on NetBSD projects, and 5 on DragonFly BSD projects.

Keep an eye on the FreeBSD wiki and mailing lists for more information about these projects over the summer. Congratulations to Robert Watson, Brooks Davis, and Tim Kientzle for pairing up mentors and students for this year's FreeBSD applications :

• Student: alejandro pulver
  Project: Ports license infrastructure (part 2: integration)
  Mentor: Erwin Lansing


• Student: Ana Kukec
  Project: IPv6 Secure Neighbor Discovery - native kernel APIs for FreeBSD
  Mentor: Bjoern A. Zeeb


• Student: sylvestre gallon
  Project: USB improvements under FreeBSD
  Mentor: Philip Paeps


• Student: David Forsythe
  Project: Package tools rewrite via a new package library, with new features
  Mentor: Tim Kientzle


• Student: Fabio Checconi
  Project: Geom-based Disk Schedulers
  Mentor: Luigi Rizzo


• Student: Fang Wang
  Project: Implement TCP UTO
  Mentor: Rui Paulo


• Student: Gabor Kovesdan
  Project: BSD-licensed libiconv in base system
  Mentor: Xin LI


• Student: Ilias Marinos
  Project: Application-Specific Audit Trails
  Mentor: Robert Watson


• Student: marta carbone
  Project: Ipfw and dummynet improvements
  Mentor: Luigi Rizzo


• Student: Prashant Vaibhav
  Project: Reworking the callout scheme: towards a tickless kernel
  Mentor: Ed Maste


• Student: nikhil bysani
  Project: Porting NetworkManager to FreeBSD
  Mentor: Ed Schouten


• Student: gabor janos pali
  Project: Design and Implementation of Subsystem Support Libraries for Monitoring and Management
  Mentor: Oleksandr Tymoshenko


• Student: Satish Srinivasan
  Project: TrustedBSD Audit: Developing BSD licensed tools for importing, exporting from/to Linux audit log format and BSM.
  Mentor: Stacey Son


• Student: tatsiana severyna
  Project: puffs (pass-to-userspace framework file system) port for FreeBSD
  Mentor: Konstantin Belousov


• Student: edward napierala
  Project: Hierarchical Resource Limits
  Mentor: Brooks Davis


• Student: aditya sarawgi
  Project: Improving Second Extended File system (ext2fs) and making it GPL free.
  Mentor: Ulf Lilleengen


• Student: Zhao Shuai
  Project: FIFO Optimizations
  Mentor: John Baldwin


• Student: Zachariah Riggle
  Project: TCP\IP Regression Testing Suite
  Mentor: George Neville-Neil


• Student: gleb kurtsov
  Project: In kernel stackable cryptographic filesystem (ownfs)
  Mentor: Stanislav Sedov


• Student: tatsiana elavaya
  Project: ipfw ruleset optimization and highlevel rule definition language
  Mentor: Diomidis Spinellis

Videos from DCBSDCon Posted

Thanks to Jason Dixon and Will Backman, the first 8 videos from the first DCBSDCon are now available in the BSDConferences YouTube channel. The audio quality for these is better than many of the previous conference videos because Jason was able to sync the audio with a direct recording from the podium taken by Will. These videos were also made with pure open source tools such as avidemux, mplayer/mencoder and audacity. More information will be coming soon to the VideoProductionAndPublishing wiki.

In the mean time, enjoy these technical presentations from DCBSDCon 2009:

• Ken Caruso, Using BSD in SchmooCon Labs
  
• Robert Luciani, M:N Threading in DragonFly BSD
  
• Kurt Miller, Implementing pie on OpenBSD
  
• Isolating Cluster Jobs for Performance and Predictability, Brooks Davis
  
• Epitome, Marco Peereboom
  
• Process Isolation for NetBSD and OpenBSD, Kristaps Dzonson
  
• OpenBSD vs SMP, Threading, and Concurrency, Ted Unangst
  
• A Narrative History of BSD, Dr. Kirk McKusick
  

Cell phones and BSD

Xin LI pointed me at BSDroid, a project which aims to provide native binaries for tools and make it possible to develop Android applications on FreeBSD. This reminded me of the underreported fact that the Android software for the G1 phone uses various bits of NetBSD and OpenBSD userland, and of course the iPhone 3g includes elements of the BSD subsystem. You can even get a terminal app for the Android. What other phones have significant BSD software in them?

All Videos from AsiaBSDCon 2008 Posted

Thanks to Hiroki Sato, we have finished uploading all remaining videos from last year's AsiaBSDCon conference. The new videos are especially appreciated because they include some NetBSD and OpenBSD videos which have previously been under represented in the bsdconferences channel and it is nice to encourage cross-fertilization of ideas (including to more general open source conferences).

• Sleeping Beauty - NetBSD on Modern Laptops, Jörg Sonnenb


• OpenBSD Network Stack Internals, Claudio Jeker


• Keynote: Hideki Sunahara - 25 years with BSD


• A Portable iSCSI Initiator, Alistair Crooks


• Logical Resource Isolation in the NetBSD Kernel, Kristaps Džonsons


• BSD Implementations of XCAST6, Yuji Imai


• Tracking FreeBSD in a Commercial Setting M. Warner Losh

These videos are in addition to the 6 other FreeBSD videos already posted from AsiaBSDCon 2008 as announced here and here.

Summer of Code 2009

By this point I hope everyone has seen the announcement from Tim Kientzle that FreeBSD will be participating in the Google Summer of Code for the fifth consecutive summer. More general information is available from the Google Open Source Blog Post about GSoC 2009.

The list of potential project ideas has improved significantly in recent weeks and is still being updated. This year Robert and Manolis put together some posters that can be used on university campuses to raise awareness about this program. Please help forward the announcement mails on to student lists and if you have a university affiliation please consider posting the posters on department bulletin boards.

A Brief History of the BSD Fast Filesystem

Two new videos are available from AsiaBSDCon 2008, including :

* A Brief History of the BSD Fast Filesystem, Kirk McKusick.
* Using FreeBSD to Promote Open Source Development Methods, Brooks Davis

It has been less than two months since I posted Dr. Kirk McKusick's Kernel Internals video and the popularity of this video has been stunning. The 11,000 views in less than 2 months makes it by far the most popular video in the bsdconferences channel.

New Summer of Code Administrators

Brooks Davis and Tim Kientzle have volunteered to help Robert Watson organize the FreeBSD participation in the Google Summer of Code program this year. After 4 consecutive years of running the program with Robert, I wanted to step aside to let others organize things this year. Brooks and Tim were excellent mentors in previous years and had lots of good ideas for improving the experience for mentors and students alike so I wish them well and am confident we'll have another successful Summer of Code. Please send all of your summer of code inquiries to soc-admins@ rather than me personally, and don't forget to spread the word about this program among potential mentors and students. Brooks, Robert, and Tim will follow up with more details about the timeline, etc.

Videos from AsiaBSDCon 2008

Thanks to Hiroki Sato we have a number of interesting FreeBSD videos available now from AsiaBSDCon 2008.

• Reducing Lock Contention in a Multi-Core System, Randall Stewart
  
• ISC and *BSD at the Core of the Internet: Keynote, Peter Losher
  
• PC-BSD, Matt Olander
  
• GEOM: In Infrastructure We Trust, Pawel Jakub Dawidek
  

Additional videos from this and other conferences are on the way. I realize it was nearly a year ago but we are just now starting to improve our processes for recording and publishing lectures at BSD conferences and should be much more timely in the future.

FreeBSD Kernel Internals Lecture Posted

The first lecture from Kirk McKusick's full length FreeBSD Kernel Internals course has been posted to the BSD Conferences channel on YouTube. It's been about 10 years since I first took a shortened version of this course at FreeBSDCon 1999, and only a few years since I took the follow up kernel code reading course in Berkeley, and I highly recommend this unique resource to others.



This makes the 24th video uploaded to the BSD Conferences channel since I created it just over a month ago. Thanks to Julian Elisher, Jason Dixon, Tomasz Dudzisz, and Kirk McKusick for uploading the conference videos and for contributing to our growing page of tips about video production and publishing on the FreeBSD Wiki.

As of this writing we have 644 unique subscribers to the channel and approximately 400 daily views of these videos. To date the most popular videos have been Kris Kennaway speaking about the New features in FreeBSD 7 at MeetBSD 2007, and Jason Dixon's tongue-in-cheek BSD is Dying talk at NYCBSDCon 2006. Note to conference organizers: high level talks about the new features, or talks by speakers as entertaining as Jason Dixon are likely to be well received. The YouTube analytics to the right show the top 10 most popular videos from the channel as well as some demographic information.

Follow FreeBSD on Twitter and send tweets from FreeBSD command line

It just came to my attention that Eric Anderson setup a FreeBSD feed on twitter. There you can find updates from the FreeBSD website, from the blogs aggregated at FreeBSD Planet, and other FreeBSD related RSS feeds published as 140 character tweets with tinyurl links to the full posts. I've been using twitter for a while now for two quite separate purposes. Primarily, I enjoy following people like Tim O'Reilly to get an endless stream of interesting tech links, ideas, and thoughts throughout the day. The updates are 140 characters or less and I only click through to those that I have time for so I find it less of a time sink than logging into my feedreader (Google Reader) and really digging into the news I'm interested in. I also find it quite useful for arranging social engagements. I use it as an SMS broadcast medium to make plans and arrange to meetup with friends for dinner, drinks, movies, or whatever after work. For the latter purpose Twitter works best in conjunction with a GPS-enabled smartphone and something like Loopt.

Following Eric's lead I setup a couple of more specific FreeBSD related twitter accounts using Twitter Feed to automatically publish the updates from RSS. The first account freebsdannounce consists of all the RSS feeds from the main www.freebsd.org website (most of which I added almost exactly one year ago). The second account freebsdblogs consists of the FreeBSD Planet combined RSS feed. If you want everything subscribe to Eric's main FreeBSD feed, but if you want only a subset of that content subscribe to one of my two more specific feeds.

Finally, I couldn't find a way to make simple updates to twitter from the base FreeBSD system command line so I created a patch for very basic HTTP POST support for fetch. Apply this patch, rebuild and reinstall libfetch(3) and fetch(1) and then you can update twitter from the command line (or send a simple POST request to other web services) with :

$ fetch -x status='Experimenting with Twitter API.' http://twitter.com/statuses/update.xml

fetch(1) will then prompt you for the HTTP authentication credentials of your twitter account.

I'm not sure how useful other people find HTTP POST support in fetch. If you would find this useful let me know and maybe I'll clean up the patch above and send it out for review.