%% Portability from a Debian perspective %% Copyright 2003 Bdale Garbee %% %% This talk will tackle the general theme that solving portability problems %% improves software quality, by examining how (and why!) Debian handles the %% "10k+ packages on 11 architectures" problem. It will explain in detail how %% the Debian distributed autobuilder network works, and will discuss how %% Debian packaging policy, the Debian archive management tools, and the %% Debian bug tracking system interact with the autobuilder network. The talk %% will conclude by exposing classes of frequently observed build failures, %% with speculation on why they happen and how they can be avoided. %% %% Define the fonts we will use %% %deffont "standard" tfont "Arial.ttf" %deffont "typewriter" tfont "Times_New_Roman.ttf" %deffont "fixed" xfont "terminal" %% %% Default settings for special lines %% %default 1 leftfill, fore "black", back "white", bimage "background.bmp" %default 2 size 7, vgap 10, prefix " ", font "standard" %default 3 size 2, bar "gray70", vgap 10 %default 4 size 5, vgap 30, font "standard" %% %% Default settings for indented lines %% %tab 1 size 5, vgap 40, prefix " ", icon box "green" 50 %tab 2 size 4, vgap 40, prefix " ", icon arc "red" 50 %tab 3 size 3, vgap 40, prefix " ", icon delta3 "blue" 40 %% %%%%%%%%%%%%%%%%%% %page %nodefault, font "standard", fore "black", back "white", bimage "background.bmp" %center Software Portability A Debian Perspective %size 6 Bdale Garbee %size 5 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page My Objectives for Today Solving portability problems improves software quality! Autobuilding as a driver on portability Debian supports more packages on more architectures... Problems Facing Debian Porters How Developers and Package Maintainers Can Help A few horrors... %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page A Few Debian Statistics Debian is a project of Software in the Public Interest 11 released architectures alpha, arm, hppa, i386, ia64, m68k, mips/mipsel, powerpc, s390, sparc other ports in progress (?) Hitachi SuperH AMD x86-64 only distribution also working with HURD & BSD kernels 1225 people 7794 source packages 12946 binary packages %%%%%%%%%%%%%%%%%% %page Number of Developers Over Time %center, newimage "developers.png" %%%%%%%%%%%%%%%%%% %page Map of Debian Developer Locations %center, image "developers.map.jpeg" 0 130 130 1 %%%%%%%%%%%%%%%%%% %page Package Count Over Time %center, newimage "packages.png" %%%%%%%%%%%%%%%%%% %page What Porting Involves Tool chain Compilers Binary Utilities Libraries Kernel User Space Applications Installation Toolkit Need a "Can-Do" Attitude Lots and lots of Iteration %%%%%%%%%%%%%%%%%% %page Tools Debian Uses for Porting Packaging System Granularity Strong Dependencies Policies Build Daemons Sbuild Minimal Chroot Environments IRC %%%%%%%%%%%%%%%%%% %page buildd.debian.org %center, newimage "buildd.png" %%%%%%%%%%%%%%%%%% %page soundmodem on buildd.debian.org %center, newimage "soundmodem.png" %%%%%%%%%%%%%%%%%% %page Bug Tracking System and Policy Manual Since 1994 Debian has had a public bugtracking system on the Internet. This is an open system to which people can submit bug reports, features requests and other comments. It is used primarily email-in, web-out. One important success factor for Debian is the integration of all the different packages. This is coordinated through the policy document, which gives guidelines for building and maintaining packages. %%%%%%%%%%%%%%%%%% %page A Day (or Less!) in the Life of a Package Someone makes a source code change Debian package is updated and uploaded Archive maintenance tools perform basic checks, package shows up in "accepted" queue Autobuilders for each architecture notice package, attempt to build Autobuilder emails log to admin and buildd.debian.org Admins process emails Successfully built packages uploaded Root causes of failures investigated, bugs filed %%%%%%%%%%%%%%%%%% %page Autobuilder Admin Tasks Chroot Maintenance Dealing With Build Daemon Email Successful Given Back Not For Us Failed for Build Dependency Failed for Source Problem Actually Fixing Portability Problems... %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page So, What's the Problem? %size 8 Fundamentally, programmers are... %pause, fore "red" %size 20 %center LAZY %size 6 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page So, What's the Problem? %center %size 8 In my experience, most portability problems come from laziness, expressed as ambiguity. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page So, What's the Problem? Not All Modern CPUs are 32 bits! ia64 is LP64 -> ints 32 bits, longs + ptrs 64 bits New Archs MUST Track New Kernels, Toolchain Revs ia64 used gcc-2.96, hppa used gcc-3.X 2.2 vs 2.4 kernels, installer issues Mixing PIC and non-PIC Code at Runtime a linker issue? sloppiness is not a virtue! Not All Platforms Have All Features serial consoles no sound hardware, joysticks, etc %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page How to Help Implicit Declarations are BAD! Own or borrow a second architecture Experience life on a non-default platform Learn first-hand why programs don't compile or run Help keep Debian vital on all these architectures! Test Builds in a Minimal CHROOT Environment only way to really verify build-depends info correct autobuilders start with essential+build-essential pbuilder by Junichi Uekawa makes this pretty easy %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Future Work Triggered autobuilding of "experimental" Better packaging of tools for use by other projects Improvements to chroot maintenance Optimal build ordering Scratch building Building subsets for custom distributions %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page For More Information... %center %size 10 http://buildd.debian.org/ testdrive.hp.com %size 8 http://www.gag.com/~bdale/talks/