%% Portability from a Debian perspective %% Copyright 2003 Bdale Garbee %% %% This talk will tackle the general theme that solving portability problems %% improves software quality, by examining how (and why!) Debian handles the %% "10k+ packages on 11 architectures" problem. It will explain in detail how %% the Debian distributed autobuilder network works, and will discuss how %% Debian packaging policy, the Debian archive management tools, and the %% Debian bug tracking system interact with the autobuilder network. The talk %% will conclude by exposing classes of frequently observed build failures, %% with speculation on why they happen and how they can be avoided. %% %% Define the fonts we will use %% %deffont "standard" tfont "Arial.ttf" %deffont "typewriter" tfont "Times_New_Roman.ttf" %deffont "fixed" xfont "terminal" %% %% Default settings for special lines %% %default 1 leftfill, fore "black", back "white", bimage "background.bmp" %default 2 size 7, vgap 10, prefix " ", font "standard" %default 3 size 2, bar "gray70", vgap 10 %default 4 size 5, vgap 30, font "standard" %% %% Default settings for indented lines %% %tab 1 size 5, vgap 40, prefix " ", icon box "green" 50 %tab 2 size 4, vgap 40, prefix " ", icon arc "red" 50 %tab 3 size 3, vgap 40, prefix " ", icon delta3 "blue" 40 %% %%%%%%%%%%%%%%%%%% %page %nodefault, font "standard", fore "black", back "white", bimage "background.bmp" %center Software Portability A Debian Perspective %size 6 Bdale Garbee HP Linux CTO %size 5 %%%%%%%%%%%%%%%%% %page Who am I? First contribution to what we now call the Free Software community in 1979 (1802 assembly language hack) First Unix login in January 1983 - Vax 4.1bsd at CMU Joined Debian project in 1995 after choosing Debian as the development platform for an AMSAT project Instigated or drove ports of Debian to 5 architectures (alpha, sparc, arm, hppa, ia64) Served as Debian Project Leader Working on Linux full-time for HP since May 2001 HP Enablement Kit for Linux on Itanium Systems HP Linux CTO since June 2003 %%%%%%%%%%%%%%%%%% %page My Objectives for Today Solving portability problems improves software quality! Autobuilding as a driver on portability Debian supports over 13,000 packages on 11 architectures Problems Debian Porters Faced The relationship between HP and Debian A few horrors... %%%%%%%%%%%%%%%%% %page What is Debian? %font "typewriter" `The Debian Project is an association of individuals who have made common cause to create a free operating system.' %font "standard" More specifically, Debian: is one of the most popular GNU/Linux distributions runs on the most architectures is a completely open volunteer organization is quite international is the largest GNU/Linux distribution works with companies to develop and maintain software is Focused on Freedom is well represented in the Free Software world %%%%%%%%%%%%%%%%%% %page A Few Debian Statistics Debian is a project of Software in the Public Interest 11 released architectures alpha, arm, hppa, i386, ia64, m68k, mips/mipsel, powerpc, s390, sparc other ports in progress (?) Hitachi SuperH AMD Opteron only distribution also working with HURD & BSD kernels 1237 people 7887 source packages 13131 binary packages %%%%%%%%%%%%%%%%%% %page Number of Developers Over Time %center, newimage "developers.png" %%%%%%%%%%%%%%%%%% %page Map of Debian Developer Locations %center, image "developers.map.jpeg" 0 130 130 1 %%%%%%%%%%%%%%%%%% %page What Porting Involves Porting is the process of making software written for one operating environment work in another environment. Tool chain Compilers Binary Utilities Libraries Kernel User Space Applications Installation Toolkit Need a "Can-Do" Attitude Lots and lots of Iteration %%%%%%%%%%%%%%%%%% %page Tools Debian Uses for Porting Packaging System Granularity Strong Dependencies Policies Capture Best Packaging Practices Tools (Lintian, Linda) to Check Compliance Bug Tracking System Since 1994, Completely Open, Email In and Web Out Autobuilders Centralized Database of Package Status Distributed Network of Build Machines Minimal Chroot Environments IRC Characteristics of Email and Instant Messaging Active Subset of Debian Developer Community %%%%%%%%%%%%%%%%%% %page Package Count Over Time %center, newimage "packages.png" %%%%%%%%%%%%%%%%%% %page A Day (or Less!) in the Life of a Package Someone makes a source code change Debian package is updated and uploaded Archive maintenance tools perform basic checks, package shows up in "accepted" queue Autobuilders for each architecture notice package, attempt to build Autobuilder emails log to admin and buildd.debian.org Admins process emails Successfully built packages uploaded Root causes of failures investigated, bugs filed %%%%%%%%%%%%%%%%%% %page buildd.debian.org %center, newimage "buildd.png" %%%%%%%%%%%%%%%%%% %page soundmodem on buildd.debian.org %center, newimage "soundmodem.png" %%%%%%%%%%%%%%%%%% %page Package Promotion towards Release All new package uploads go to the "unstable" (sid) tree. Packages which meet certain criteria are promoted to the "testing" (sarge) tree. Package ages 2 weeks. No release critical bugs. All installation dependencies met. Security updates immediately available. Periodic release cycles. Major releases every 12-24 months. 3.0 (woody) currently. Point releases with security fixes every 2 months. %%%%%%%%%%%%%%%%%% %page So, What's the Problem? %pause %center %size 8 In my experience, most portability problems come from laziness, expressed as ambiguity. %size 6 %%%%%%%%%%%%%%%%%% %page So, What's the Problem? Not All Modern CPUs are 32 bits! ia64 is LP64 -> ints 32 bits, longs + ptrs 64 bits Implicit Declarations are BAD! New Archs MUST Track New Kernels, Toolchain Revs ia64 used gcc-2.96, hppa used gcc-3.X 2.2 vs 2.4 kernels, installer issues Mixing PIC and non-PIC Code at Runtime a linker issue? sloppiness is not a virtue! Not All Platforms Have All Features serial consoles no sound hardware, joysticks, etc %%%%%%%%%%%%%%%%%% %page Why HP Picked Debian PARISC was One of the Last 32-bit CPUs to Run Linux Kernel Port Irrelevant Without a Distribution HP Thinks of PARISC Port as a Community Activity Learning How to Work With/In Free Software Community Hacker Driven, not Business Driven Commercial Distributors Want "Real Money" Debian Perceived as a "Commons" Community Development Model Instantiated No Single Business In Control Companies can Invest w/o Contracts Ability to Influence Future Direction Debian Developers have Credibility Upstream Key Engineers at HP and Linuxcare Liked Debian %%%%%%%%%%%%%%%%%% %page How HP Uses Debian GNU/Linux Development Platform Hardware Enablement Most Itanium 2 systems ran Debian as their first OS boot! Early hardware shipments. Carrier-grade servers. Blade servers. Benchmarks - Itanium 2 SPECfp2000 HP Enablement Kit for Linux on Itanium Systems. iPAQ distributions are Debian derived. familiar - minimal Linux environment intimate - full Debian for iPAQ Infrastructure HP's relationship with Debian helps HP stay "connected" with the Free Software community. %%%%%%%%%%%%%%%%%% %page Summary %font "typewriter" Portability problems often represent ambiguity in source code... fixing them improves the overall quality of the software. Debian's acceptance of the PARISC port led to a mutually beneficial relationship between the project and the company. %font "standard" %%%%%%%%%%%%%%%%%% %page For More Information... %center %size 6 %size 8 http://www.debian.org/ports/ http://buildd.debian.org/ %size 6 %size 8 http://www.hp.com/linux/ http://opensource.hp.com/ http://testdrive.hp.com/ http://www.gag.com/~bdale/talks/