Download Embedded Linux Primer: A Practical, Real-World
Transcript
ChristopherHallinan ComprehensiveReal-WorldGuidanceforEveryEmbeddedDeveloperandEngineer This book brings together indispensable knowledge for building efficient, high-value, Linux-based embedded products: information that has never been assembled in one place before. Drawing on years of experience as an embedded Linux consultant and field application engineer, Christopher Hallinan offers solutions for the specific technical issues you're most likelytoface,demonstrateshowtobuildaneffectiveembeddedLinuxenvironment,andshowshowtouseitasproductivelyas possible. HallinanbeginsbytouringatypicalLinux-basedembeddedsystem,introducingkeyconceptsandcomponents,andcalling attention to differences between Linux and traditional embedded environments. Writing from the embedded developer's viewpoint,hethoroughlyaddressesissuesrangingfromkernelbuildingandinitializationtobootloaders,devicedriverstofile systems. HallinanthoroughlycoverstheincreasinglypopularBusyBoxutilities;presentsastep-by-stepwalkthroughofportingLinux to custom boards; and introduces real-time configuration via CONFIG_RT--one of today's most exciting developments in embeddedLinux.You'llfindespeciallydetailedcoverageofusingdevelopmenttoolstoanalyzeanddebugembeddedsystems-includingtheartofkerneldebugging. •CompareleadingembeddedLinuxprocessors •UnderstandthedetailsoftheLinuxkernelinitializationprocess •LearnaboutthespecialroleofbootloadersinembeddedLinuxsystems,withspecificemphasisonU-Boot •UseembeddedLinuxfilesystems,includingJFFS2--withdetailedguidelinesforbuildingFlash-residentfilesystemimages •UnderstandtheMemoryTechnologyDevicessubsystemforflash(andother)memorydevices •Mastergdb,KGDB,andhardwareJTAGdebugging •LearnmanytipsandtechniquesfordebuggingwithintheLinuxkernel •Maximizeyourproductivityincross-developmentenvironments •Prepareyourentiredevelopmentenvironment,includingTFTP,DHCP,andNFStargetservers •Configure,build,andinitializeBusyBoxtosupportyouruniquerequirements PrenticeHallOpenSourceSoftwareDevelopmentSeries ArnoldSeriesEditorRobbins "Realworldcodefromrealworldapplications" OpenSourcetechnologyhasrevolutionizedthecomputingworld.Manylarge-scaleprojectsare in production use worldwide, such as Apache, MySQL, and Postgres, with programmers writing applicationsinavarietyoflanguagesincludingPerl,Python,andPHP.Thesetechnologiesareinuse onmanydifferentsystems,rangingfromproprietarysystems,toLinuxsystems,totraditionalUNIX systems,tomainframes. ThePrenticeHallOpenSourceSoftwareDevelopmentSeriesisdesignedtobringyouthebestof theseOpenSourcetechnologies.Notonlywillyoulearnhowtousethemforyourprojects,butyou willlearnfromthem.Byseeingrealcodefromrealapplications,youwilllearnthebestpracticesof OpenSourcedeveloperstheworldover. Titlescurrentlyintheseriesinclude: Linux®DebuggingandPerformanceTuning:TipsandTechniques SteveBest 0131492470,Paper,©2006 UnderstandingAJAX:UsingJavaScripttoCreateRichInternetApplications JoshuaEichorn 0132216353,Paper,©2007 EmbeddedLinuxPrimer ChristopherHallinan 0131679848,Paper,©2007 SELinuxbyExample FrankMayer,DavidCaplan,KarlMacMillan 0131963694,Paper,©2007 UNIXtoLinux®Porting AlfredoMendoza,ChakaratSkawratananond,ArtisWalker 0131871099,Paper,©2006 LinuxProgrammingbyExample:TheFundamentals ArnoldRobbins 0131429647,Paper,©2004 TheLinux®KernelPrimer:ATop-DownApproachforx86andPowerPCArchitectures ClaudiaSalzberg,GordonFischer,StevenSmolski 0131181637,Paper,©2006 Foreword Computersareeverywhere. Thisfact,ofcourse,isnotasurprisetoanyonewhohasn'tbeenlivinginacaveduringthepast25 yearsorso.Andyouprobablyknowthatcomputersaren'tjustonourdesktops,inourkitchens,and, increasingly, in our living rooms holding our music collections. They're also in our microwave ovens,ourregularovens,ourcellphones,andourportabledigitalmusicplayers. And if you're holding this book, you probably know a lot, or are interested in learning more about,theseembeddedcomputersystems. Untilnottoolongago,embeddedsystemswerenotverypowerful,andtheyranspecial-purpose, proprietaryoperatingsystemsthatwereverydifferentfromindustry-standardones.(Plus,theywere much harder to develop for.) Today, embedded computers are as powerful as, if not more than, a modernhomecomputer.(Considerthehigh-endgamingconsoles,forexample.) Alongwiththispowercomesthecapabilitytorunafull-fledgedoperatingsystemsuchasLinux. UsingasystemsuchasLinuxforanembeddedproductmakesalotofsense.Alargecommunityof developers are making it possible. The development environment and the deployment environment canbesurprisinglysimilar,whichmakesyourlifeasadevelopermucheasier.Andyouhaveboththe securityofaprotectedaddressspacethatavirtualmemory-basedsystemgivesyou,andthepower andflexibilityofamultiuser,multiprocesssystem.That'sagooddealallaround. Forthisreason,companiesallovertheworldareusingLinuxonmanydevicessuchasPDAs, homeentertainmentsystems,andeven,believeitornot,cellphones! I'm excited about this book. It provides an excellent "guide up the learning curve" for the developerwhowantstouseLinuxforhisorherembeddedsystem.It'sclear,well-written,andwellorganized;Chris'sknowledgeandunderstandingshowthroughateveryturn.It'snotonlyinformative andhelpfulit'salsoenjoyabletoread. Ihopeyoubothlearnsomethingandhavefunatthesametime.IknowIdid. ArnoldRobbins SeriesEditor Preface AlthoughmanygoodbookscoverLinux,nonebringstogethersomanydimensionsofinformation andadvicespecificallytargetedtotheembeddedLinuxdeveloper.Indeed,therearesomeverygood books written about the Linux kernel, Linux system administration, and so on. You will find references right here in this book to many of the ones that I consider to be at the top of their categories. MuchofthematerialpresentedinthisbookismotivatedbyquestionsI'vereceivedovertheyears fromdevelopmentengineers,inmycapacityasanembeddedLinuxconsultantandmypresentroleas a Field Application Engineer for Monta Vista Software, the leading vendor of embedded Linux distributions. Embedded Linux presents the experienced software engineer with several unique challenges. First,thosewithmanyyearsofexperiencewithlegacyreal-timeoperatingsystems(RTOSes)findit difficult to transition their thinking from those environments to Linux. Second, experienced application developers often have difficulty understanding the relative complexities of a crossdevelopmentenvironment. Althoughthisisa primer,intended fordevelopersnewtoembeddedLinux,Iamconfidentthat evendeveloperswhoareexperiencedinembeddedLinuxwillfindsomeusefultipsandtechniques thatIhavelearnedovertheyears. PracticalAdviceforthePracticingEmbeddedDeveloper Thisbookcontainsmyviewofwhatanembeddedengineerneedstoknowtogetuptospeedfast inanembeddedLinuxenvironment.InsteadoffocusingonLinuxkernelinternals,thekernelchapter inthisbookfocusesontheprojectnatureofthekernelandleavestheinternalstotheotherexcellent texts on the subject. You will learn the organization and layout of the kernel source tree. You will discover the binary components that make up a kernel image, and how they are loaded and what purpose they serve on an embedded system. One of my favorite figures in the book is Figure 5-1, whichschematicallyillustratesthebuildprocessofacompositekernelimage. Inthepagesofthisbook,youwilllearnhowthebuildsystemworksandhowtoincorporateinto theLinuxkernelyourowncustomchangesthatarerequiredforyourownprojects.Youwilldiscover themechanismusedtodrivetheconfigurationofdifferentarchitecturesandfeatureswithintheLinux kernel source tree and, more important, how to modify this system to customize it to your own requirements. We also cover in detail the kernel command-line mechanism. You will learn how it works,howtoconfigurethekernel'sruntimebehaviorforyourrequirements,andhowtoextendthis functionalitytoyourownproject.Youwilllearnhowtonavigatethekernelsourcecodeandhowto configurethekernelforspecifictasksrelatedtoanembeddedsystem.Youwilllearnmanyusefultips andtricksforyourembeddedproject,frombootloaders,systeminitialization,filesystems,andFlash memorytoadvancedkernel-andapplication-debuggingtechniques. IntendedAudience ThisbookisintendedforprogrammerswithaworkingknowledgeofprogramminginC.Iassume that you have a rudimentary understanding of local area networks and the Internet. You should understand and recognize an IP address and how it is used on a simple local area network. I also assume that you have an understanding of hexadecimal and octal numbering systems, and their commonusageinatextsuchasthis. SeveraladvancedconceptsrelatedtoCcompilingandlinkingareexplored,soyouwillbenefit from having at least a cursory understanding of the role of the linker in ordinary C programming. KnowledgeoftheGNUmakeoperationandsemanticswillalsoprovebeneficial. WhatThisBookIsNot This book is not a detailed hardware tutorial. One of the difficulties the embedded developer faces is the huge variety of hardware devices in use today. The user manual for a modern 32-bit processorwithsomeintegratedperipheralscaneasilyexceed1,000pages.Therearenoshortcuts.If youneedtounderstandahardwaredevicefromaprogrammer'spointofview,youwillneedtospend plentyofhours in yourfavoritereadingchairwithhardwaredatasheetsandreferenceguides,and manymorehourswritingandtestingcodeforthesehardwaredevices! ThisisalsonotabookabouttheLinuxkernelorkernelinternals.Inthisbook,youwon'tlearn about the intricacies of the Memory Management Unit (MMU) used to implement Linux's virtual memory-managementpoliciesandprocedures;therearealreadyseveralgoodbooksonthissubject. Youareencouragedtotakeadvantageofthe"SuggestionsforAdditionalReading"sectionfoundat theendofeverychapter. ConventionsUsed Filenames and code statements are presented in Courier. Commands issued by the reader are indicatedinboldCourier.Newtermsorimportantconceptsarepresentedinitalics. Whenyouseeapathnameprecededwiththreedots,thisreferencesawell-knownbutunspecified top-level directory. The top-level directory is context dependent but almost universally refers to a top-level Linux source directory. For example, .../arch/ppc/kernel/setup.c refers to the setup.c file located in the architecture branch of a Linux source tree. The actual path might be something like ~/sandbox/linux.2.6.14/arch/ppc/kernel/setup.c. OrganizationoftheBook Chapter1,"Introduction,"providesabrieflookatthefactorsdrivingtherapidadoptionofLinux intheembeddedenvironment.Severalimportantstandardsandorganizationsrelevanttoembedded Linuxareintroduced. Chapter2,"YourFirstEmbeddedExperience,"introducesthereadertomanyconceptsrelatedto embeddedLinuxuponwhichwebuildinlaterchapters. InChapter3,"ProcessorBasics,"wepresentahigh-levellookatthemorepopularprocessors andplatformsthatarebeingusedtobuildembeddedLinuxsystems.Weexamineselectedproducts from many of the major processor manufacturers. All of the major architecture families are represented. Chapter 4, "The Linux Kernel: A Different Perspective," examines the Linux kernel from a slightly different perspective. Instead of kernel theory or internals, we look at its structure, layout, and build construction so you can begin to learn your way around this large software project and, moreimportant,learnwhereyourowncustomizationeffortsmustbefocused.Thisincludesdetailed coverageofthekernelbuildsystem. Chapter5,"KernelInitialization,"detailstheLinuxkernel'sinitializationprocess.Youwilllearn howthearchitecture-andbootloader-specificimagecomponentsareconcatenatedtotheimageofthe kernel proper for downloading to Flash and booting by an embedded bootloader. The knowledge gained here will help you customize the Linux kernel to your own embedded application requirements. Chapter6,"SystemInitialization,"continuesthedetailedexaminationoftheinitializationprocess. When the Linux kernel has completed its own initialization, application programs continue the initialization process in a predetermined manner. Upon completing Chapter 6, you will have the necessaryknowledgetocustomizeyourownuserlandapplicationstartupsequence. Chapter 7, "Bootloaders," is dedicated to the booloader and its role in an embedded Linux system.Weexaminethepopularopen-sourcebootloaderU-Bootandpresentaportingexample.We brieflyintroduceadditionalbootloadersinusetodaysoyoucanmakeaninformedchoiceaboutyour particularrequirements. Chapter 8, "Device Driver Basics," introduces the Linux device driver model and provides enoughbackgroundtolaunchintooneofthegreattextsondevicedrivers,listedas"Suggestionsfor AdditionalReading"attheendofthechapter. Chapter 9, "File Systems," presents the more popular file systems being used in embedded systemstoday.WeincludecoverageoftheJFFS2,animportantembeddedfilesystemusedonFlash memorydevices.Thischapterincludesabriefintroductiononbuildingyourownfilesystemimage, oneofthemoredifficulttaskstheembeddedLinuxdeveloperfaces. Chapter 10, "MTD Subsystem," explores the Memory Technology Devices (MTD) subsystem. MTDisanextremelyusefulabstractionlayerbetweentheLinuxfilesystemandhardwarememory devices,primarilyFlashmemory. Chapter 11, "BusyBox," introduces BusyBox, one of the most useful utilities for building small embedded systems. We describe how to configure and build BusyBox for your particular requirements,alongwithdetailedcoverageofsysteminitializationuniquetoaBusyBoxenvironment. AppendixC,"BusyBoxCommands,"liststheavailableBusyBoxcommandsfromarecentBusyBox release. Chapter 12, "Embedded Development Environment," takes a detailed look at the unique requirements of a typical cross-development environment. Several techniques are presented to enhance your productivity as an embedded developer, including the powerful NFS root mount developmentconfiguration. Chapter13,"DevelopmentTools,"examinesmanyusefuldevelopmenttools.Debuggingwithgdb is introduced, including coverage of core dump analysis. Many more tools are presented and explained,withexamplesincludingstrace,ltrace,top,andps,andthememoryprofilersmtraceand dmalloc.Thechapterconcludeswithanintroductiontothemoreimportantbinaryutilities,including thepowerfulreadelfutility. Chapter14,"KernelDebuggingTechniques,"providesadetailedexaminationofmanydebugging techniquesusefulfordebugginginsidetheLinuxkernel.Weintroducetheuseofthekerneldebugger KGDB, andpresentmanyusefuldebugging techniquesusing thecombinationof gdbandKGDB as debugging tools. Included is an introduction to using hardware JTAG debuggers and some tips for analyzingfailureswhenthekernelwon'tboot. Chapter 15, "Debugging Embedded Linux Applications," moves the debugging context from the kerneltoyourapplicationprograms.Wecontinuetobuildonthegdbexamplesfromtheprevioustwo chapters,andwepresenttechniquesformultithreadedandmultiprocessdebugging. Chapter16,"PortingLinux,"introducestheissuesrelatedtoportingLinuxtoyourcustomboard. WewalkthroughasimpleexampleandhighlightthestepstakentoproduceaworkingLinuxkernelon a custom PowerPC board. Several important concepts are presented that have trapped many newcomerstoLinuxkernelporting.TogetherwiththetechniquespresentedinChapters13and14, youshouldbereadytotackleyourowncustomboardportafterreadingthischapter. Chapter 17, "Linux and Real Time," provides an introduction to one of the more exciting developmentsinembeddedLinux:configuringforrealtimeviatheCONFIG_RToption.Wecover thefeaturesavailablewithRTandhowtheycanbeusedinadesign.Wealsopresenttechniquesfor measuringlatencyinyourapplicationconfiguration. The appendixes cover the GNU Public License, U-Boot Configurable Commands, BusyBox Commands, SDRAM Interface Considerations, resources for the open source developer, and a sampleconfigurationfileforoneofthemorepopularhardwareJTAGdebuggers,theBDI-2000. FollowAlong Youwillbenefitmostfromthisbookifyoucandivideyourtimebetweenthepagesofthisbook and your favorite Linux workstation. Grab an old x86 computer to experiment on an embedded system.Evenbetter,ifyouhaveaccesstoasingle-boardcomputerbasedonanotherarchitecture,use that.Youwillbenefitfromlearningthelayoutandorganizationofaverylargecodebase(theLinux kernel),andyouwillgainsignificantknowledgeandexperienceasyoupokearoundthekerneland learnbydoing. Look at the code and try to understand the examples produced in this book. Experiment with different settings, configuration options, and hardware devices. Much can be gained in terms of knowledge,andbesides,it'sloadsoffun! GPLCopyrightNotice Portions of open-source code reproduced in this book are copyrighted by a large number of individualandcorporatecontributors.Thecodereproducedherehasbeenlicensedundertheterms oftheGNUPublicLicenseorGPL. AppendixAcontainsthetextoftheGNUGeneralPublicLicense. Acknowledgments Iamconstantlyamazedbythegraciousnessofopensourcedevelopers.Iamhumbledbythetalent inourcommunitythatoftenfarexceedsmyown.Duringthecourseofthisproject,Ireachedoutto manypeopleintheLinuxandopensourcecommunitywithquestions.Mostoftenmyquestionswere quicklyansweredandwithencouragement.Innoparticularorder,I'dliketoexpressmygratitudeto thefollowingmembersoftheLinuxandopensourcecommunitywhohavecontributedanswerstomy questions: DanMalek providedinspiriationforsomeofthecontentsofChapter2,"YourFirst Embedded Experience." DanKegelandDanielJacobowitzpatientlyansweredmytoolchainquestions. ScottAndersonprovidedtheoriginalideasforthegdbmacrospresentedinChapter14,"Kernel DebuggingTechniques." BradDixoncontinuestochallengeandexpandmytechnicalvisionthroughhisown. GeorgeDavisansweredmyARMquestions. JimLewisprovidedcommentsandsuggestionsontheMTDcoverage. CalEricksonansweredmygdbusequestions. JohnTwomeyadvisedonChapter3,"ProcessorBasics." LeeRevell,Sven-ThorstenDietrich,andDanielWalkeradvisedonrealtimeLinuxcontent. Many thanks toAMCC, Embedded Planet,UltimateSolutions,andUnitedElectronicIndustries for providing hardware for the examples. Many thanks to my employer, Monta Vista Software, for tolerating the occasional distraction and for providing software for some of the examples. Many others contributed ideas, encouragement, and support over the course of the project. To them I am alsograteful. Iwishtoacknowledgemysincereappreciationtomyprimaryreviewteam,whopromptlyread each chapter and provided excellent feedback, comments, and ideas. Thank you Arnold Robbins, SandyTerrace,KurtLloyd,andRobFarber.ManythankstoArnoldforhelpingthisnewbielearnthe ropesofwritingatechnicalbook.Whileeveryattempthasbeenmadetoeliminatemistakes,those thatremainaresolelymyown. IwanttothankMarkL.Taubforbringingthisprojecttofruitionandforhisencouragementand infinitepatience!IwishtothanktheproductionteamincludingKristyHart,JenniferCramer,Krista Hansing,andCherylLenser. Andfinally,averyspecialandheartfeltthankyoutoCaryDillmanwhoreadeachchapterasit waswritten,andforherconstantencouragementandheroccasionalsacrificethroughouttheproject. ChrisHallinan AbouttheAuthor ChristopherHallinaniscurrentlyfieldapplicationsengineerforMontaVistaSoftware,livingand workinginMassachusetts.Chrishasspentmorethan25yearsinthenetworkingandcommunications marketplacemostlyinvariousproductdevelopmentroles,wherehedevelopedastrongbackground inthespacewherehardwaremeetssoftware.PriortojoiningMontaVista,Chrisspentfouryearsas an independent Linux consultant providing custom Linux board ports, device drivers, and bootloaders. Chris's introduction to the open source community was through contributions to the popularU-Bootbootloader.WhennotmessingaboutwithLinux,heisoftenfoundsingingandplaying aTaylororMartin. Chapter1.Introduction The move away from proprietary operating systems is causing quite a stir in the corporate boardroomsofmanytraditionalembeddedoperatingsystem(OS)companies.Formanywell-founded reasons, Linux is being adopted as the operating system in many products beyond its traditional stronghold in server applications. Examples of these embedded systems include cellular phones, DVDplayers,video games,digitalcameras,networkswitches,andwirelessnetworkinggear.Itis quitepossiblethatLinuxisalreadyinyourhomeoryourautomobile. 1.1.WhyLinux? Because of the numerous economic and technical benefits, we are seeing strong growth in the adoption of Linux for embedded devices. This trend has crossed virtually all markets and technologies. Linux has been adopted for embedded products in the worldwide public switched telephonenetwork,globaldatanetworks,wirelesscellularhandsets,andtheequipmentthatoperates these networks. Linux has enjoyed success in automobile applications, consumer products such as gamesandPDAs,printers,enterpriseswitchesandrouters,andmanyotherproducts.Theadoption rateofembeddedLinuxcontinuestogrow,withnoendinsight. SomeofthereasonsforthegrowthofembeddedLinuxareasfollows: •Linuxhasemergedasamature,high-performance,stablealternativetotraditionalproprietary embeddedoperatingsystems. •Linuxsupportsahugevarietyofapplicationsandnetworkingprotocols. • Linux is scalable, from small consumer-oriented devices to large, heavy-iron, carrier-class switchesandrouters. • Linux can be deployed without the royalties required by traditional proprietary embedded operatingsystems. • Linux has attracted a huge number of active developers, enabling rapid support of new hardwarearchitectures,platforms,anddevices. • An increasing number of hardware and software vendors, including virtually all the top-tier manufacturersandISVs,nowsupportLinux. Fortheseandotherreasons,weareseeinganacceleratedadoptionrateofLinuxinmanycommon householditems,rangingfromhigh-definitiontelevisionsetstocellularhandsets. 1.2.EmbeddedLinuxToday It might come as no surprise that Linux has experienced significant growth in the embedded space.Indeed,thefactthatyouarereadingthisbookindicatesthatithastouchedyourownlife.Itis difficulttoestimatethemarketsizebecausemanycompaniesstillbuildtheirownembeddedLinux distributions. LinuxDevices.com,thepopularnewsandinformationportalfoundedbyRichLehrbaum,conducts an annual survey of the embedded Linux market. In its latest survey, they report that Linux has emergedasthedominantoperatingsystemusedinthousandsofnewdesignseachyear.Infact,nearly half of respondents reported using Linux in an embedded design, while the nearest competing operating system was reportedly used by only about one in every eight respondents. Commercial operatingsystemsthatoncedominatedtheembeddedmarketwerereportedlyusedbyfewerthanone intenrespondents.Evenifyoufindreasontodisputetheseresults,noonecanignorethemomentum intheembeddedLinuxmarketplacetoday. 1.3.OpenSourceandtheGPL OneofthefundamentalfactorsdrivingtheadoptionofLinuxisthefactthatitisopensource.The LinuxkernelislicensedunderthetermsoftheGNUGPL[1](GeneralPublicLicense),whichleadsto thepopularmyththatLinuxisfree.[2]Infact,thesecondparagraphoftheGNUGPLdeclares:"When wespeakoffreesoftware,wearereferringtofreedom,notprice."TheGPLlicenseisremarkably shortandeasytoread.Amongthemostimportantkeycharacteristics: •Thelicenseisself-perpetuating. •Thelicensegrantstheuserfreedomtoruntheprogram. •Thelicensegrantstheusertherighttostudyandmodifythesourcecode. •Thelicensegrantstheuserpermissiontodistributetheoriginalcodeorhismodifications. •ThelicensegrantsthesesamerightstoanyonetowhomyoudistributeGPLsoftware. WhenasoftwareworkisreleasedunderthetermsoftheGPL,itmustforevercarrythatlicense. [3] Even if the code is highly modified, which is allowed and even encouraged by the license, the GPL mandates that it must be released under the same license. The intent of this feature is to guaranteeaccesstoeveryone,evenofmodifiedversionsofthesoftware(orderivedworks,asthey arecommonlycalled). No matter how the software was obtained, the GPL grants the licensee unlimited distribution rights,withouttheobligationtopayroyaltiesorper-unitfees.Thisdoesnotmeanthatavendorcan't chargefortheGPLsoftwarethisisaveryreasonablecommonbusinesspractice.Itmeansthatoncein possessionofGPLsoftware,itispermissibletomodifyandredistributeit,whetheritisaderived (modified)workornot.However,asdefinedbytheGPLlicense,theauthor(s)ofthemodifiedwork areobligatedtoreleasetheworkunderthetermsoftheGPLiftheydecidetodoso.Anydistribution ofaderivedwork,suchasshipmenttoacustomer,triggersthisobligation. Forafascinatingandinsightfullookatthehistoryandcultureoftheopensourcemovement,read EricS.Raymond'sbookreferencedattheendofthischapter. 1.3.1.FreeVersusFreedom Twopopularphrasesareoftenrepeatedinthediscussionaboutthefreenatureofopensource: "free asinfreedom" and"freeasinbeer."(Theauthorisparticularlyfondofthelatter.)TheGPL licenseexiststoguarantee"freeasinfreedom"ofaparticularbodyofsoftware.Itguaranteesyour freedomtouseit,studyit,andchangeit.Italsoguaranteesthesefreedomsforanyonetowhomyou distributeyourmodifiedcode.Thisconcepthasbecomefairlywidelyunderstood. OneofthemisconceptionsfrequentlyheardisthatLinuxis"freeasinbeer."Sure,youcanobtain Linuxfreeofcost.YoucandownloadaLinuxkernelinafewminutes.However,asanyprofessional developmentmanagerunderstands,certaincostsareassociatedwithanysoftwaretobeincorporated into a design. These include the costs of acquisition, integration, modification, maintenance, and support.Addtothatthecostofobtainingandmaintainingaproperlyconfiguredtoolchain,libraries, application programs, and specialized cross-development tools compatible with your chosen architecture,andyoucanquicklyseethatitisanontrivialexercisetodeveloptheneededsoftware componentstodeployyourembeddedLinux-basedsystem. 1.4.StandardsandRelevantBodies As Linux continues to gain market share in the desktop, enterprise, and embedded market segments,newstandardsandorganizationsareemergingtohelpinfluencetheuseandacceptanceof Linux.Thissectionservesasaresourcetointroducethestandardsthatyoumightwanttofamiliarize yourselfwith. 1.4.1.LinuxStandardBase Probably the single most relevant standard is the Linux Standard Base (LSB). The goal of the LSBistoestablishasetofstandardsdesignedtoenhancetheinteroperabilityofapplicationsamong different Linux distributions. Currently, the LSB spans several architectures, including IA32/64, PowerPC32-and64-bit,AMD64,andothers.Thestandardisbrokendownintoacorecomponent andtheindividualarchitecturecomponents. The LSB specifies common attributes of a Linux distribution, including object format, standard library interfaces, minimum set of commands and utilities and their behavior, file system layout, systeminitialization,andsoon. YoucanlearnmoreabouttheLSBatthelinkgiveninSection1.5.1,"SuggestionsforAdditional Reading,"sectionattheendofthischapter. 1.4.2.OpenSourceDevelopmentLabs OpenSourceDevelopmentLabs(OSDL)wasformedtohelpacceleratetheacceptanceofLinux inthegeneralmarketplace.Accordingtoitsmissionstatement,OSDLcurrentlyprovidesenterpriseclasstestingfacilitiesandothertechnicalsupporttotheLinuxcommunity.Ofsignificance,OSDLhas sponsoredseveralworkinggroupstodefinestandardsandparticipateinthedevelopmentoffeatures targetingthreeimportantmarketsegments.Thenextthreesectionsintroducetheseinitiatives. 1.4.2.1.OSDL:CarrierGradeLinux A significant number of the world's largest networking and telecommunications equipment manufacturers are either developing or shipping carrier-class equipment running Linux as the operating system. Significant features of carrier-class equipment include high reliability, high availability, and rapid serviceability. These vendors design products using redundant, hot-swap architectures,fault-tolerantfeatures,clustering,andoftenreal-timeperformance. The OSDL Carrier Grade Linux working group has produced a specification defining a set of requirements for carrier-class equipment. The current version of the specification covers seven functionalareas: • Availability Requirements that provide enhanced availability, including online maintenance operations,redundancy,andstatusmonitoring • Clusters Requirements that facilitate redundant services, such as cluster membership managementanddatacheckpointing • Serviceability Requirements for remote servicing and maintenance, such as SNMP and diagnosticmonitoringoffansandpowersupplies • Performance Requirements to define performance and scalability, symmetric multiprocessing, latencies,andmore •StandardsRequirementsthatdefinestandardstowhichCGL-compliantequipmentshallconform • Hardware Requirements related to high-availability hardware, such as blade servers and hardware-managementinterfaces •SecurityRequirementstoimproveoverallsystemsecurityfromvariousthreats 1.4.2.2.OSDL:MobileLinuxInitiative As this book is written, several mobile handsets (cellular phones) are available on the worldwide market that have been built around embedded Linux. It has been widely reported that millionsofhandsetshavebeenshippedbasedonLinux.Theonlycertaintyisthatmorearecoming. Thispromisestobeoneofthemostexplosivemarketsegmentsforwhatwasformerlytheroleofa proprietary real-time operating system. This speaks volumes about the readiness of Linux for commercialembeddedapplications. TheOSDLsponsorsaworkinggroupcalledMobileLinuxInitiative.Itspurposeistoaccelerate the adoption of Linux on next-generation mobile handsets and other converged voice/data portable devices, according to the OSDL website. The areas of focus for this working group include development tools, I/O and networking, memory management, multimedia, performance, power management,security,andstorage. 1.4.2.3.ServiceAvailabilityForum Ifyou are engagedinbuildingproductsforenvironmentsinwhichhighreliability, availability, andserviceability(RAS)areimportant,youshouldbeawareoftheServiceAvailabilityForum(SA Forum).Thisorganizationisplayingaleadingroleindefiningacommonsetofinterfacesforusein carrier-grade and other commercial equipment for system management. The SA Forum website is www.saforum.org. 1.5.ChapterSummary • Adoption of Linux among developers and manufacturers of embedded products continues to accelerate. •UseofLinuxinembeddeddevicescontinuestogrowatanexcitingpace. • In this chapter, we present many of the factors driving the growth of Linux in the embedded market. • Several standards and relevant organizations influencing embedded Linux were presented in thischapter. 1.5.1.SuggestionsforAdditionalReading TheCathedralandtheBazaar EricS.Raymond O'ReillyMedia,Inc.,2001 LinuxStandardBaseProject www.linuxbase.org OpenSourceDevelopmentLabs,Inc. www.osdl.org Chapter2.YourFirstEmbeddedExperience Oftenthebestpathtounderstandingagiventaskistohaveagoodgraspofthebigpicture.Many fundamental concepts can present challenges to the newcomer to embedded systems development. This chapter takes you on a tour of a typical embedded system and the development environment, withspecificemphasisontheconceptsandcomponentsthatmakedevelopingthesesystemsunique andoftenchallenging. 2.1.EmbeddedorNot? Several key attributes are usually associated with embedded systems. We wouldn't necessarily callourdesktopPCanembeddedsystem.ButconsideradesktopPChardwareplatforminaremote data center that is performing a critical monitoring and alarm task. Assume that this data center is normally not staffed. This imposes a different set of requirements on this hardware platform. For example, if power is lost and then restored, we would expect this platform to resume its duties withoutoperatorintervention. Embedded systems come in a variety of shapes and sizes, from the largest multiple-rack data storage or networking powerhouses to tiny modules such as your personal MP3 player or your cellularhandset.Someoftheusualcharacteristicsofembeddedsystemsincludethese: •Containaprocessingengine,suchasageneral-purposemicroprocessor •Typicallydesignedforaspecificapplicationorpurpose •Includesasimple(orno)userinterfaceanautomotiveengineignitioncontroller,forexample •Oftenisresourcelimitedforexample,hasasmallmemoryfootprintandnoharddrive •Mighthavepowerlimitations,suchasarequirementtooperatefrombatteries •Usuallyisnotusedasageneral-purposecomputingplatform •Generallyhasapplicationsoftwarebuiltin,notuserselected •Shipswithallintendedapplicationhardwareandsoftwarepreintegrated •Oftenisintendedforapplicationswithouthumanintervention Most commonly, embedded systems are resource constrained compared to the typical desktop PC. Embedded systems often have limited memory, small or no hard drives, and sometimes no external network connectivity. Frequently, the only user interface is a serial port and some LEDs. Theseandotherissuescanpresentchallengestotheembeddedsystemdeveloper. 2.1.1.BIOSVersusBootloader When power is first applied to the desktop computer, a software program called the BIOS immediately takes control of the processor. (Historically, BIOS was an acronym meaning Basic Input/Output Software, but the acronym has taken on a meaning of its own because the functions it performs have become much more complex than the original implementations.) The BIOS might actually be stored in Flash memory (described shortly), to facilitate field upgrade of the BIOS programitself. TheBIOSisacomplexsetofsystem-configurationsoftwareroutinesthathaveknowledgeofthe low-leveldetailsofthehardwarearchitecture.MostofusareunawareoftheextentoftheBIOSand itsfunctionality,butitisacriticalpieceofthedesktopcomputer.TheBIOSfirstgainscontrolofthe processorwhenpowerisapplied.Itsprimaryresponsibilityistoinitializethehardware,especially thememorysubsystem,andloadanoperatingsystemfromthePC'sharddrive. In a typical embedded system (assuming that it is not based on an industry-standard x86 PC hardwareplatform)abootloaderisthesoftwareprogramthatperformsthesesamefunctions.Inyour own custom embedded system, part of your development plan must include the development of a bootloaderspecifictoyourboard.Luckily,severalgoodopensourcebootloadersareavailablethat youcancustomizeforyourproject.TheseareintroducedinChapter7,"Bootloaders." Someofthemoreimportanttasksthatyourbootloaderperformsonpower-upareasfollows: • Initializes criticalhardwarecomponents,suchastheSDRAMcontroller,I/Ocontrollers,and graphicscontrollers •Initializessystemmemoryinpreparationforpassingcontroltotheoperatingsystem •Allocatessystemresourcessuchasmemoryandinterruptcircuitstoperipheralcontrollers,as necessary •Providesamechanismforlocatingandloadingyouroperatingsystemimage •Loadsandpassescontroltotheoperatingsystem,passinganyrequiredstartupinformationthat might be required, such as total memory size clock rates, serial port speeds and other low-level hardwarespecificconfigurationdata This is a very simplified summary of the tasks that a typical embedded-system bootloader performs. The important point to remember is this: If your embedded system will be based on a custom-designedplatform,thesebootloaderfunctionsmustbesuppliedbyyou,thesystemdesigner.If your embedded system is based on a commercial off-the-shelf (COTS) platform such as an ATCA chassis,[4]typicallythebootloader(andoftentheLinuxkernel)isincludedontheboard.Chapter7 discussesbootloadersindetail. 2.2.AnatomyofanEmbeddedSystem Figure2-1showsablockdiagramofatypicalembeddedsystem.Thisisaverysimpleexample ofahigh-levelhardwarearchitecturethatmightbefoundinawirelessaccesspoint.Thesystemis centeredona32-bitRISCprocessor.Flashmemoryisusedfornonvolatileprogramanddatastorage. Main memory is synchronous dynamic random-access memory (SDRAM) and might contain anywherefromafewmegabytestohundredsofmegabytes,dependingontheapplication.Areal-time clock module, often backed up by battery, keeps the time of day (calendar/wall clock, including date). This example includes an Ethernet and USB interface, as well as a serial port for console accessviaRS-232.The802.11chipsetimplementsthewirelessmodemfunction. Figure2-1.Exampleembeddedsystem OftentheprocessorinanembeddedsystemperformsmanyfunctionsbeyondthetraditionalCPU. The hypothetical processor in Figure 2-1 contains an integrated UART for a serial interface, and integratedUSBandEthernetcontrollers.Manyprocessorscontainintegratedperipherals.Welookat severalexamplesofintegratedprocessorsinChapter3,"ProcessorBasics." 2.2.1.TypicalEmbeddedLinuxSetup OftenthefirstquestionposedbythenewcomertoembeddedLinuxis,justwhatdoesoneneedto begin development? To answer that question, we look at a typical embedded Linux development setup(seeFigure2-2). Figure2-2.EmbeddedLinuxdevelopmentsetup Here we show a very common arrangement. We have a host development system, running your favoritedesktopLinuxdistribution,suchasRedHatorSuSEorDebianLinux.OurembeddedLinux target board is connected to the development host via an RS-232 serial cable. We plug the target board'sEthernetinterfaceintoalocalEthernethuborswitch,towhichourdevelopmenthostisalso attachedviaEthernet.Thedevelopmenthostcontainsyourdevelopmenttoolsandutilitiesalongwith targetfilesnormallyobtainedfromanembeddedLinuxdistribution. For this example, our primary connection to the embedded Linux target is via the RS-232 connection.Aserialterminalprogramisusedtocommunicatewiththetargetboard.Minicomisone of the most commonly used serial terminal applications and is available on virtually all desktop Linuxdistributions. 2.2.2.StartingtheTargetBoard When power is first applied, a bootloader supplied with your target board takes immediate control of the processor. It performs some very low-level hardware initialization, including processorandmemorysetup,initializationoftheUARTcontrollingtheserialport,andinitialization oftheEthernetcontroller.Listing2-1displaysthecharactersreceivedfromtheserialport,resulting from power being applied to the target. For this example, we have chosen a target board from AMCC, the PowerPC 440EP Evaluation board nicknamed Yosemite. This is basically a reference design containing the AMCC 440EP embedded processor. It ships from AMCC with the U-Boot bootloaderpreinstalled. Listing2-1.InitialBootloaderSerialOutput U-Boot1.1.4(Mar182006-20:36:11) AMCCPowerPC440EPRev.B Board:Yosemite-AMCCPPC440EPEvaluationBoard VCO:1066MHz CPU:533MHz PLB:133MHz OPB:66MHz EPB:66MHz PCI:66MHz I2C:ready DRAM:256MB FLASH:64MB PCI:BusDevVenIdDevIdClassInt In:serial Out:serial Err:serial Net:ppc_4xx_eth0,ppc_4xx_eth1 => 2.3.StorageConsiderations Oneofthemostchallengingaspectsofembeddedsystemsisthatmostembeddedsystemshave limitedphysicalresources.AlthoughthePentium4machineonyourdesktopmighthave180GBof hard drive space, it is not uncommon to find embedded systems with a fraction of that amount. In manycases,theharddriveistypicallyreplacedbysmallerandlessexpensivenonvolatilestorage devices. Hard drives are bulky, have rotating parts, are sensitive to physical shock, and require multiplepowersupplyvoltages,whichmakesthemunsuitableformanyembeddedsystems. 2.3.1.FlashMemory NearlyeveryoneisfamiliarwithCompactFlashmodules[5]usedinawidevarietyof consumer devices, such as digital cameras and PDAs (both great examples of embedded systems). These modules can be thought of as solid-state hard drives, capable of storing many megabytesand even gigabytesofdatainatinyfootprint.Theycontainnomovingparts,arerelativelyrugged,andoperate onasinglecommonpowersupplyvoltage. Several manufacturers of Flash memory exist. Flash memory comes in a variety of physical packagesandcapacities.Itisnotuncommontoseeembeddedsystemswithaslittleas1MBor2MB of nonvolatile storage. More typical storage requirements for embedded Linux systems range from 4MBto256MBormore.AnincreasingnumberofembeddedLinuxsystemshavenonvolatilestorage intothegigabyterange. Flash memory can be written to and erased under software control. Although hard drive technology remains the fastest writable media, Flash writing and erasing speeds have improved considerablyoverthecourseoftime,thoughflashwriteanderasetimeisstillconsiderablyslower. SomefundamentaldifferencesexistbetweenharddriveandFlashmemorytechnologythatyoumust understandtoproperlyusethetechnology. Flashmemoryisdividedintorelativelylargeerasableunits,referredtoaseraseblocks.Oneof thedefiningcharacteristicsofFlashmemoryisthewayinwhichdatainFlashiswrittenanderased. InatypicalFlashmemorychip,datacanbechangedfromabinary1toabinary0undersoftware control,1bit/wordatatime,buttochangeabitfromazerobacktoaone,anentireblockmustbe erased.Blocksareoftencallederaseblocksforthisreason. A typical Flash memory device contains many erase blocks. For example, a 4MB Flash chip mightcontain64eraseblocksof64KBeach.Flashmemoryisalsoavailablewithnonuniformerase blocksizes,tofacilitateflexibledata-storagelayout.Thesearecommonlyreferredtoasbootblock orbootsectorFlashchips.Oftenthebootloaderisstoredinthesmallerblocks,andthekerneland otherrequireddataarestoredinthelargerblocks.Figure2-3illustratestheblocksizelayoutfora typicaltopbootFlash. Figure2-3.Bootblockflasharchitecture TomodifydatastoredinaFlashmemoryarray,theblockinwhichthemodifieddataresidesmust becompletelyerased.Evenifonly1byteinablockneedstobechanged,theentireblockmustbe erased and rewritten.[6] Flash block sizes are relatively large, compared to traditional hard-drive sector sizes. In comparison, a typical high-performance hard drive has writable sectors of 512 or 1024 bytes. The ramifications of this might be obvious: Write times for updating data in Flash memorycanbemanytimesthatofaharddrive,dueinparttotherelativelylargequantityofdatathat mustbewrittenbacktotheFlashforeachupdate.Thesewritecyclescantakeseveralseconds,inthe worstcase. AnotherlimitationofFlashmemorythatmustbeconsideredisFlashmemorycellwritelifetime. A Flash memory cell has a limited number of write cycles before failure. Although the number of cyclesisfairlylarge(100Kcyclestypicalperblock),itiseasytoimagineapoorlydesignedFlash storagealgorithm(orevenabug)thatcanquicklydestroyFlashdevices.Itgoeswithoutsayingthat youshouldavoidconfiguringyoursystemloggerstooutputtoaFlash-baseddevice. 2.3.2.NANDFlash NANDFlashisarelativelynewFlashtechnology.WhenNANDFlashhitthemarket,traditional FlashmemorysuchasthatdescribedintheprevioussectionwasreferredtoasNORFlash.These distinctionsrelatetotheinternalFlashmemorycellarchitecture.NANDFlashdevicesimproveupon someofthelimitationsoftraditional(NOR)Flashbyofferingsmallerblocksizes,resultinginfaster andmoreefficientwritesandgenerallymoreefficientuseoftheFlasharray. NORFlashdevicesinterfacetothemicroprocessorinafashionsimilartomanymicroprocessor peripherals. That is, they have a parallel data and address bus that are connected directly[7] to the microprocessor data/address bus. Each byte or word in the Flash array can be individually addressedinarandomfashion.Incontrast,NANDdevicesareaccessedseriallythroughacomplex interface that varies among vendors. NAND devices present an operational model more similar to thatofatraditionalharddriveandassociatedcontroller.Dataisaccessedinserialbursts,whichare far smaller than NOR Flash block size. Write cycle lifetime for NAND Flash is an order of magnitudegreaterthanforNORFlash,althougherasetimesaresignificantlysmaller. Insummary,NORFlashcanbedirectlyaccessedbythemicroprocessor,andcodecanevenbe executeddirectlyoutofNORFlash(though,forperformancereasons,thisisrarelydone,andthen only on systems in which resources are extremely scarce). In fact, many processors cannot cache instruction accesses to Flash like they can with DRAM. This further impacts execution speed. In contrast, NAND Flash is more suitable for bulk storage in file system format than raw binary executablecodeanddatastorage. 2.3.3.FlashUsage An embedded system designer has many options in the layout and use of Flash memory. In the simplest of systems, in which resources are not overly constrained, raw binary data (perhaps compressed)canbestoredontheFlashdevice.Whenbooted,afilesystemimagestoredinFlashis readintoaLinuxramdiskblockdevice,mountedasafilesystemandaccessedonlyfromRAM.This isoftenagooddesignchoicewhenthedatainFlashrarelyneedstobeupdated,andanydatathat does need to be updated is relatively small compared to the size of the ramdisk. It is important to realizethatanychangestofilesintheramdiskarelostuponrebootorpowercycle. Figure2-4illustratesacommonFlashmemoryorganizationthatistypicalofasimpleembedded systeminwhichnonvolatilestoragerequirementsofdynamicdataaresmallandinfrequent. Figure2-4.ExampleFlashmemorylayout The bootloader is often placed in the top or bottom of the Flash memory array. Following the bootloader,spaceisallocatedfortheLinuxkernelimageandtheramdiskfilesystemimage,[8]which holds the root file system. Typically, the Linux kernel and ramdisk file system images are compressed,andthebootloaderhandlesthedecompressiontaskduringthebootcycle. Fordynamicdatathatneedstobesavedbetweenrebootsandpowercycles,anothersmallareaof Flash can be dedicated, or another type of nonvolatile storage[9] can be used. This is a typical configurationforembeddedsystemswithrequirementstostoreconfigurationdata,asmightbefound inawirelessaccesspointaimedattheconsumermarket,forexample. 2.3.4.FlashFileSystems ThelimitationsofthesimpleFlashlayoutschemedescribedinthepreviousparagraphscanbe overcomebyusingaFlashfilesystemtomanagedataontheFlashdeviceinamannersimilartohow dataisorganizedonaharddrive.EarlyimplementationsoffilesystemsforFlashdevicesconsisted of a simple block device layer that emulated the 512-byte sector layout of a common hard drive. These simple emulation layers allowed access to data in file format rather than unformatted bulk storage,buttheyhadsomeperformancelimitations. OneofthefirstenhancementstoFlashfilesystemswastheincorporationofwearleveling.As discussed earlier, Flash blocks are subject to a finite write lifetime. Wear-leveling algorithms are usedtodistributewritesevenlyoverthephysicaleraseblocksoftheFlashmemory. AnotherlimitationthatarisesfromtheFlasharchitectureistheriskofdatalossduringapower failure or premature shutdown. Consider that the Flash block sizes are relatively large and that average file sizes being written are often much smaller relative to the block size. You learned previously that Flash blocksmustbewrittenoneblockatatime.Therefore,towriteasmall8KB file,youmusteraseandrewriteanentireFlashblock,perhaps64KBor128KBinsize;intheworst case,thiscantaketensofsecondstocomplete.Thisopensasignificantwindowofriskofdataloss duetopowerfailure. One of the more popular Flash file systems in use today is JFFS2, or Journaling Flash File System2.Ithasseveralimportantfeaturesaimedatimprovingoverallperformance,increasingFlash lifetime, and reducing the risk of data loss in case of power failure. The more significant improvements in the latest JFFS2 file system include improved wear leveling, compression and decompression to squeeze more data into a given Flash size, and support for Linux hard links. We coverthisindetailinChapter9,"FileSystems,"andagaininChapter10,"MTDSubsystem,"when wediscusstheMemoryTechnologyDevice(MTD)subsystem. 2.3.5.MemorySpace Virtually all legacy embedded operating systems view and manage system memory as a single large, flat address space. That is, a microprocessor's address space exists from 0 to the top of its physicaladdressrange.Forexample,ifamicroprocessorhad24physicaladdresslines,itstopof memory would be 16MB. Therefore, its hexadecimal address would range from 0x00000000 to 0x00ffffff.HardwaredesignscommonlyplaceDRAMstartingatthebottomoftherange,andFlash memoryfromthetopdown.UnusedaddressrangesbetweenthetopofDRAMandbottomofFLASH wouldbeallocatedforaddressingofvariousperipheralchipsontheboard.Thisdesignapproachis oftendictatedbythechoiceofmicroprocessor.Figure2-5isanexampleofatypicalmemorylayout forasimpleembeddedsystem. Figure2-5.Typicalembeddedsystemmemorymap Intraditionalembeddedsystemsbasedonlegacyoperatingsystems,theOSandallthetasks[10] hadequalaccessrightstoallresourcesinthesystem.Abuginoneprocesscouldwipeoutmemory contents anywhere in the system, whether it belonged to itself, the OS, another task, or even a hardwareregistersomewhereintheaddressspace.Althoughthisapproachhadsimplicityasitsmost valuablecharacteristic,itledtobugsthatcouldbedifficulttodiagnose. High-performance microprocessors contain complex hardware engines called Memory ManagementUnits(MMUs)whosepurposeistoenableanoperatingsystemtoexerciseahighdegree of management and control over its address space and the address space it allocates to processes. Thiscontrolcomesintwoprimaryforms:accessrightsandmemorytranslation.Accessrightsallow an operating system to assign specific memory-access privileges to specific tasks. Memory translationallowsanoperatingsystemtovirtualizeitsaddressspace,whichhasmanybenefits. TheLinuxkerneltakesadvantageofthesehardwareMMUstocreateavirtualmemoryoperating system. One of the biggest benefits of virtual memory is that it can make more efficient use of physicalmemorybypresentingtheappearancethatthesystemhasmorememorythan isphysically present.Theotherbenefitisthatthekernelcanenforceaccessrightstoeachrangeofsystemmemory thatitallocatestoataskorprocess,topreventoneprocessfromerrantlyaccessingmemoryorother resourcesthatbelongtoanotherprocessortothekernelitself. Let's look at some details of how this works. A tutorial on the complexities of virtual memory systems is beyond the scope of this book.[11] Instead, we examine the ramifications of a virtual memorysystemasitappearstoanembeddedsystemsdeveloper. 2.3.6.ExecutionContexts OneoftheveryfirstchoresthatLinuxperformswhenitbeginstorunistoconfigurethehardware memorymanagementunit(MMU)ontheprocessorandthedatastructuresusedtosupportit,andto enable address translation. When this step is complete, the kernel runs in its own virtual memory space. The virtual kernel address selected by the kernel developers in recent versions defaults to 0xC0000000. In most architectures, this is a configurable parameter.[12] If we were to look at the kernel'ssymboltable,wewouldfindkernelsymbolslinkedatanaddressstartingwith0xC0xxxxxx. As a result, any time the kernel is executing code in kernel space, the instruction pointer of the processorwillcontainvaluesinthisrange. In Linux, we refer to two distinctly separate operational contexts, based on the environment in which a given thread[13] is executing. Threads executing entirely within the kernel are said to be operatinginkernelcontext,whileapplicationprogramsaresaidtooperateinuserspacecontext.A user space process can access only memory it owns, and uses kernel system calls to access privilegedresourcessuchasfileanddeviceI/O.Anexamplemightmakethismoreclear. Consider an application that opens a file and issues a read request (see Figure 2-6). The read functioncallbeginsinuserspace,intheClibraryread()function.TheClibrarythenissuesaread request to the kernel. The read request results in a context switch from the user's program to the kernel,toservicetherequestforthefile'sdata.Insidethekernel,thereadrequestresultsinaharddriveaccessrequestingthesectorscontainingthefile'sdata. Figure2-6.Simplefilereadrequest Usuallythehard-drivereadisissuedasynchronouslytothehardwareitself.Thatis,therequestis posted to the hardware, and when the data is ready, the hardware interrupts the processor. The applicationprogramwaitingforthedataisblockedonawaitqueueuntilthedataisavailable.Later, whentheharddiskhasthedataready,itpostsahardwareinterrupt.(Thisdescriptionisintentionally simplifiedforthepurposesofthisillustration.)Whenthekernelreceivesthehardwareinterrupt,it suspendswhateverprocesswasexecutingandproceedstoreadthewaitingdatafromthedrive.This isanexampleofathreadofexecutionoperatinginkernelcontext. Tosummarizethisdiscussion,wehaveidentifiedtwogeneralexecutioncontexts,userspaceand kernelspace.Whenanapplicationprogramexecutesasystemcallthatresultsinacontextswitchand entersthekernel,itisexecutingkernelcodeonbehalfofaprocess.Youwilloftenhearthisreferred toasprocesscontextwithinthekernel.Incontrast,theinterruptserviceroutine(ISR)handlingthe IDE drive (or any other ISR, for that matter) is kernel code that is not executing on behalf of any particularprocess.Severallimitationsexistinthisoperationalcontext,includingthelimitationthat the ISR cannot block (sleep) or call any kernel functions that might result in blocking. For further readingontheseconcepts,consultSection2.5.1,"SuggestionsforAdditionalReading,"attheendof thischapter. 2.3.7.ProcessVirtualMemory Whenaprocessisspawnedforexample,whentheusertypeslsattheLinuxcommandpromptthe kernel allocates memory for the process and assigns a range of virtual-memory addresses to the process. The resulting address values bear no fixed relationship to those in the kernel, nor to any other running process. Furthermore, there is no direct correlation between the physical memory addressesontheboardandthevirtualmemoryasseenbytheprocess.Infact,itisnotuncommonfor a process to occupy multiple different physical addresses in main memory during its lifetime as a resultofpagingandswapping. Listing2-4isthevenerable"HelloWorld,"asmodifiedtoillustratethepreviousconcepts.The goalwiththisexampleistoillustratetheaddressspacethatthekernelassignstotheprocess.This code was compiled and run on the AMCC Yosemite board, described earlier in this chapter. The boardcontains256MBofDRAMmemory. Listing2-4.HelloWorld,EmbeddedStyle #include<stdio.h> intbss_var;/*Uninitializedglobalvariable*/ intdata_var=1;/*Initializedglobalvariable*/ intmain(intargc,char**argv){ void*stack_var;/*Localvariableonthestack*/ stack_var=(void*)main;/*Don'tletthecompileroptimizeitout*/ printf("Hello,World!Mainisexecutingat%p\n",stack_var); printf("Thisaddress(%p)isinourstackframe\n",&stack_var); /*bsssectioncontainsuninitializeddata*/ printf("Thisaddress(%p)isinourbsssection\n",&bss_var); /*datasectioncontainsinitializateddata*/ printf("Thisaddress(%p)isinourdatasection\n",&data_var); return0; } Listing2-5showstheconsoleoutputthatthisprogramproduces.Noticethattheprocesscalled hellothinksitisexecutingsomewhereinhighRAMjustabovethe256MBboundary(0x10000418). Noticealsothatthestackaddressisroughlyhalfwayintoa32-bitaddressspace,wellbeyondour 256MBofRAM(0x7ff8ebb0).Howcanthisbe?DRAMisusuallycontiguousinsystemslikethese. Tothecasualobserver,itappearsthatwehavenearly2GBofDRAMavailableforouruse.These virtualaddresseswereassignedbythekernelandarebackedbyphysicalRAMsomewherewithin the256MBrangeofavailablememoryontheYosemiteboard. Listing2-5.HelloOutput root@amcc:~#./hello Hello,World!Mainisexecutingat0x10000418 Thisaddress(0x7ff8ebb0)isinourstackframe Thisaddress(0x10010a1c)isinourbsssection Thisaddress(0x10010a18)isinourdatasection root@amcc:~# OneofthecharacteristicsofavirtualmemorysystemisthatwhenavailablephysicalRAMgoes below a designated threshold, the kernel can swap memory pages out to a bulk storage medium, usuallyaharddiskdrive.Thekernelexaminesitsactivememoryregions,determineswhichareasin memoryhavebeenleastrecentlyused,andswapsthesememoryregionsouttodisk,tofreethemup for the current process. Developers of embedded systems often disable swapping on embedded systemsbecauseofperformanceorresourceconstraints.Forexample,itwouldberidiculousinmost casestousearelativelyslowFlashmemorydevicewithlimitedwritelifecyclesasaswapdevice. Withoutaswapdevice,youmustcarefullydesignyourapplicationstoexistwithinthelimitationsof youravailablephysicalmemory. 2.3.8.Cross-DevelopmentEnvironment Beforewecandevelopapplicationsanddevicedriversforanembeddedsystem,weneedaset oftools(compiler,utilities,andsoon)thatwillgeneratebinaryexecutablesintheproperformatfor thetargetsystem.ConsiderasimpleapplicationwrittenonyourdesktopPC,suchasthetraditional "Hello World" example. After you have created the source code on your desktop, you invoke the compiler that came with your desktop system (or that you purchased and installed) to generate a binaryexecutableimage.Thatimagefileisproperlyformattedtoexecuteonthemachineonwhichit wascompiled. Thisisreferredtoasnative compilation. That is, using compilers on your desktop system,yougeneratecodethatwillexecuteonthatdesktopsystem. Notethatnativedoesnotimplyanarchitecture.Indeed,ifyouhaveatoolchainthatrunsonyour targetboard,youcan natively compile applicationsforyourtarget'sarchitecture.Infact,onegreat waytotestanewkernelandcustomboardistorepeatedlycompiletheLinuxkernelonit. Developingsoftwareinacross-developmentenvironmentrequiresthatthecompilerrunningon yourdevelopmenthostoutputabinaryexecutablethatisincompatiblewiththedesktopdevelopment workstation on which it was compiled. The primary reason these tools exist is that it is often impractical or impossible to develop and compile software natively on the embedded system becauseofresource(typicallymemoryandCPUhorsepower)constraints. Numerous hidden traps to this approach often catch the unwary newcomer to embedded development.Whenagivenprogramiscompiled,thecompileroftenknowshowtofindincludefiles, andwheretofindlibrariesthatmightberequiredforthecompilationtosucceed.Toillustratethese concepts, let's look again at the "Hello World" program. The example reproduced in Listing 2-4 abovewascompiledwiththefollowingcommandline: gcc-Wall-ohellohello.c From Listing 2-4, we see an include the file stdio.h. This file does not reside in the same directoryasthehello.cfilespecifiedonthegcccommandline.Sohowdoesthecompilerfindthem? Also,theprintf()functionisnotdefinedinthefilehello.c.Therefore,whenhello.ciscompiled,it willcontainanunresolvedreferenceforthissymbol.Howdoesthelinkerresolvethisreferenceat linktime? Compilershavebuilt-indefaultsforlocatingincludefiles.Whenthereferencetotheincludefile isencountered,thecompilersearchesitsdefaultlistoflocationstolocatethefile.Asimilarprocess exists for the linker to resolve the reference to the external symbol printf(). The linker knows by defaulttosearchtheClibrary(libc-*)forunresolvedreferences.Again,thisdefaultbehaviorisbuilt intothetoolchain. Now consider that you are building an application targeting a PowerPC embedded system. Obviously, you will need a cross-compiler to generate binary executables compatible with the PowerPC processor architecture. If you issue a similar compilation command using your crosscompilertocompilethehello.cexampleabove,itispossiblethatyourbinaryexecutablecouldend up being accidentally linked with an x86 version of the C library on your development system, attempting to resolve the reference to printf(). Of course, the results of running this bogus hybrid executable,containingamixofPowerPCandx86binaryinstructions[14]arepredictable:crash! Thesolutiontothispredicamentistoinstructthecross-compilertolookinnonstandardlocations topickuptheheaderfilesandtargetspecificlibraries.Wecoverthistopicinmuchmoredetailin Chapter12,"EmbeddedDevelopmentEnvironment."Theintentofthisexamplewastoillustratethe differencesbetweenanativedevelopmentenvironment,andadevelopmentenvironmenttargetedat cross-compilationforembeddedsystems.Thisisbutoneofthecomplexitiesofacross-development environment. The same issue and solutions apply to cross-debugging, as you will see starting in Chapter14,"KernelDebuggingTechniques."Apropercross-developmentenvironmentiscrucialto your success and involves much more than just compilers, as we shall soon see in Chapter 12, "EmbeddedDevelopmentEnvironment." 2.4.EmbeddedLinuxDistributions Whatexactlyisadistributionanyway?AftertheLinuxkernelboots,itexpectstofindandmounta rootfilesystem.Whenasuitablerootfilesystemhasbeenmounted,startupscriptslaunchanumber ofprogramsandutilitiesthatthesystemrequires.Theseprogramsofteninvokeotherprogramstodo specific tasks, such as spawn a login shell, initialize network interfaces, and launch a user's applications.Eachoftheseprogramshasspecificrequirementsofthesystem.MostLinuxapplication programs depend on one or more system libraries. Other programs require configuration and log files, and so on. In summary, even a small embedded Linux system needs many dozens of files populatedinanappropriatedirectorystructureonarootfilesystem. Full-blown desktop systems have many thousands of files on the root file system. These files comefrompackagesthatareusuallygroupedbyfunctionality.Thepackagesaretypicallyinstalled andmanagedusingapackagemanager.RedHat'sPackageManager(rpm)isapopularexampleand is widely used for installing, removing, and updating packages on a Linux system. If your Linux workstation is based on Red Hat, including the Fedora Core series, typing rpm -qa at a command promptlistsallthepackagesinstalledonyoursystem. A package can consist of many files; indeed, some packages contain hundreds of files. A complete Linux distribution can contain hundreds or even thousands of packages. These are some examplesofpackagesthatyoumightfindonanembeddedLinuxdistribution,andtheirpurpose: •initscriptsContainsbasicsystemstartupandshutdownscripts. •apacheImplementsthepopularApachewebserver. •telnet-serverContainsfilesnecessarytoimplementtelnetserverfunctionality,whichallowsyou toestablishTelnetsessionstoyourembeddedtarget. •glibcStandardClibrary • busybox Compact versions of dozens of popular command line utilities commonly found on UNIX/Linuxsystems.[15] This is the purpose of a Linux distribution as the term has come to be used. A typical Linux distribution comes with several CD-ROMs full of useful programs, libraries, tools, utilities, and documentation.Installationofadistributiontypicallyleavestheuserwithafullyfunctionalsystem basedonareasonablesetofdefaultconfigurationoptions,whichcanbetailoredtosuitaparticular setofrequirements.YoumaybefamiliarwithoneofthepopulardesktopLinuxdistributions,suchas RedHatorSuse. A Linux distribution for embedded targets differs in several significant ways. First, the executabletargetbinariesfromanembeddeddistributionwillnotrunonyourPC,butaretargetedto the architecture and processor of your embedded system. (Of course, if your embedded Linux distributiontargetsthex86architecture,thisstatementdoesnotapply.)AdesktopLinuxdistribution tends to have many GUI tools aimed at the typical desktop user, such as fancy graphical clocks, calculators, personal time-management tools, email clients and more. An embedded Linux distributiontypicallyomitsthesecomponentsinfavorofspecializedtoolsaimedatdevelopers,such asmemoryanalysistools,remotedebugfacilities,andmanymore. Another significant difference between desktop and embedded Linux distributions is that an embedded distribution typically contains cross-tools, as opposed to native tools. For example, the gcc toolchain that ships with an embedded Linux distribution runs on your x86 desktop PC, but produces binary code that runs on your target system. Many of the other tools in the toolchain are similarlyconfigured:Theyrunonthedevelopmenthost(usuallyanx86PC)butoperateonforeign architecturessuchasARMorPowerPC. 2.4.1.CommercialLinuxDistributions ThereareseveralvendorsofcommercialembeddedLinuxdistributions.Theleadingembedded LinuxvendorshavebeenshippingembeddedLinuxdistributionsforsomeyears.Linuxdevices.com, a popular embedded Linux news and information portal, has compiled a comprehensive list of commerciallyavailableembeddedLinuxdistributions.Itissomewhatdatedbutisstillaveryuseful startingpoint.Youcanfindtheircompilationatwww.linuxdevices.com/articles/AT9952405558.html. 2.4.2.Do-It-YourselfLinuxDistributions You can choose to assemble all the components you need for your embedded project on your own. You will have to decide whether the risks are worth the effort. If you find yourself involved with embedded Linux purely for the pleasure of it, such as for a hobby or college project, this approachmightbeagoodone.However,plantospendasignificantamountoftimeassemblingall thetoolsandutilitiesyourprojectneeds,andmakingsuretheyallinteroperatetogether. For starters, you will need a toolchain. Gcc and binutils are available from www.fsf.org and othermirrorsaroundtheworld.Botharerequiredtocompilethekernelanduser-spaceapplications foryourproject.Thesearedistributedprimarilyinsourcecodeform,andyoumustcompilethetools tosuityourparticularcross-developmentenvironment.Patchesareoftenrequiredtothemostrecent "stable" source trees of these utilities, especially when they will be used beyond the x86/IA32 architecture. The patches can usually be found at the same location as the base packages. The challengeistodiscoverwhichpatchyouneedforyourparticularproblemand/orarchitecture. 2.5.ChapterSummary This chapter covered many subjects in a broad-brush fashion. Now you have a proper perspectiveforthematerialtofollowinsubsequentchapters.Inlaterchapters,thisperspectivewill beexpandedtodeveloptheskillsandknowledgerequiredtobesuccessfulinyournextembedded project. • Embedded systems share some common attributes. Often resources are limited, and user interfacesaresimpleornonexistent,andareoftendesignedforaspecificpurpose. •Thebootloaderisacriticalcomponentofatypicalembeddedsystem.Ifyourembeddedsystem isbasedonacustom-designedboard,youmustprovideabootloaderaspartofyourdesign.Often thisisjustaportingeffortofanexistingbootloader. •Severalsoftwarecomponentsarerequiredtobootacustomboard,includingthebootloaderand thekernelandfilesystemimage. •FlashmemoryiswidelyusedasastoragemediuminembeddedLinuxsystems.Weintroduced theconceptofFlashmemoryandexpandonthiscoverageinChapters9and10. •Anapplicationprogram,alsocalledaprocess,livesinitsownvirtualmemoryspaceassigned bythekernel.Applicationprogramsaresaidtoruninuserspace. •Aproperlyequippedandconfiguredcross-developmentenvironmentiscrucialtotheembedded developer.WedevoteanentirechaptertothisimportantsubjectinChapter12. • You need an embedded Linux distribution to begin development of your embedded target. Embedded distributions contain many components, compiled and optimized for your chosen architecture. 2.5.1.SuggestionsforAdditionalReading LinuxKernelDevelopment,2ndEdition RobertLove NovellPress,2005 UnderstandingtheLinuxKernel DanielP.Bovet&MarcoCesati O'Reilly&Associates,Inc.,2002 UnderstandingtheLinuxVirtualMemoryManager BrucePerens PrenticeHall,2004 Chapter3.ProcessorBasics Inthischapter,wepresentsomebasicinformationtohelpyounavigatethehugeseaofembedded processor choices. We look at some of the processors on the market and the types of features they contain.Stand-aloneprocessorsarehighlightedfirst.Thesetendtobethemostpowerfulprocessors andrequireexternalchipsetstoformcompletesystems.Nextwepresentsomeofthemanyintegrated processors that are supported under Linux. Finally, we look at some of the common hardware platformsinusetoday. Literally dozens of embedded processors are available to choose from in a given embedded design. For the purposes of this chapter, we limit the available discussion to those that contain a hardwarememory-managementunitand,ofcourse,tothosethataresupportedunderLinux.Oneof thefundamentalarchitecturaldesignaspectsofLinuxisthatitisavirtualmemoryoperatingsystem. [16]EmployingLinuxonaprocessorthatdoesnotcontainanMMUgivesuponeofthemorevaluable architecturalfeaturesofthekernelandisbeyondthescopeofthisbook. 3.1.Stand-aloneProcessors Stand-alone processors refer to processor chips that are dedicated solely to the processing function. As opposed to integrated processors, stand-alone processors require additional support circuitryfortheirbasicoperation.Inmanycases,thismeansachipsetorcustomlogicsurrounding theprocessortohandlefunctionssuchasDRAMcontroller,systembusaddressingconfiguration,and external peripheral devices such as keyboard controllers and serial ports. Stand-alone processors oftenofferthehighestoverallCPUperformance. Numerous processors exist in both 32-bit and 64-bit implementations[17] that have seen widespreaduseinembeddedsystems.TheseincludetheIBMPowerPC970FX,theIntelPentiumM, andtheFreescaleMPC74xxHostProcessors,amongothers. Herewepresentasamplefromeachofthemajormanufacturesofstand-aloneprocessors.These processorsarewellsupportedunderLinuxandhavebeenusedinmanyembeddedLinuxdesigns. 3.1.1.IBM970FX TheIBM970FXprocessorcoreisahigh-performance64-bitcapablestand-aloneprocessor.The 970FX is a superscalar architecture. This means the core is capable of fetching, issuing, and obtaining results from more than one instruction at a time. This is done through a pipelining architecture, which provides the effect of multiple streams of instruction simultaneously. The IBM 970FX contains up to 25 stages of pipelining, depending on the instruction stream and operations containedtherein. Someofthekeyfeaturesofthe970FXareasfollows: •A64-bitimplementationofthepopularPowerPCarchitecture •Deeplypipelineddesign,forvery-high-performancecomputingapplications •Staticanddynamicpower-managementfeatures •Multiplesleepmodes,tominimizepowerrequirementsandmaximizebatterylife •Dynamicallyadjustableclockrates,supportinglower-powermodes •Optimizedforhigh-performance,low-latencystoragemanagement TheIBM970FXhasbeenincorporatedintoanumberofhigh-endserverbladesandcomputing platforms,includingIBM'sownBladeServerplatform. 3.1.2.IntelPentiumM Certainly one of the most popular architectures, x86 in both 32- and 64-bit flavors (more properlycalledIA32andIA64,respectively)hasbeenemployedforembeddeddevicesinavariety ofapplications.Inthemostcommonform,theseplatformsarebasedonavarietyofcommercialoffthe-shelf (COTS) hardware implementations. Numerous manufacturers supply x86 single-board computers and complete platforms in a variety of form factors. See Section 3.2, "Integrated Processors:SystemsonChip,"laterinthischapterforadiscussionofthemorecommonplatformsin usetoday. TheIntelPentiumMhasbeenusedinawidevarietyoflaptopcomputersandhasfoundanichein embedded products. Like the IBM 970FX processor, the Pentium M is a superscalar architecture. Thesecharacteristicsmakeitattractiveinembeddedapplications: • The Pentium M is based on the popular x86 architecture, and thus is widely supported by a largeecosystemofhardwareandsoftwarevendors. •Itconsumeslesspowerthanotherx86processors. •Advancedpower-managementfeaturesenablelow-poweroperatingmodesandmultiplesleep modes. •Dynamicclockspeedcapabilityenhancesbattery-poweredoperationssuchasstandby. • On chip thermal monitoring enables automatic transition to lower power modes, to reduce powerconsumptioninovertemperatureconditions. • Multiple frequency and voltage operating points (dynamically selectable) are designed to maximizebatterylifeinportableequipment. Manyofthesefeaturesareespeciallyusefulforembeddedapplications.Itisnotuncommonfor embedded products to require portable or battery-powered configurations. The Pentium M has enjoyedpopularityinthisapplicationspacebecauseofitspower-andthermal-managementfeatures. 3.1.3.FreescaleMPC7448 The Freescale MPC7448 contains what is referred to as a fourth-generation PowerPC core, commonlycalledG4.[18]Thishigh-performance32-bitprocessoriscommonlyfoundinnetworking and telecommunications applications. Several companies manufacture blades that conform to an industry-standard platform specification, including this and other similar stand-alone Freescale processors.WeexaminetheseplatformsinSection3.3,"HardwarePlatforms,"laterinthischapter. The MPC7448 has enjoyed popularity in a wide variety of signal-processing and networking applicationsbecauseoftheadvancedfeaturesethighlightedhere: •Operatingclockratesinexcessof1.5GHz •1MBonboardL2cache •Advancedpower-managementcapabilities,includingmultiplesleepmodes •AdvancedAltiVecvector-executionunit •Voltagescalingforreduced-powerconfigurations The MPC7448 contains a Freescale technology called AltiVec to enable very fast algorithmic computations and other data-crunching applications. The AltiVec unit consists of a register file containing32verywide(128-bit)registers.EachvaluewithinoneoftheseAltiVecregisterscanbe considered a vector of multiple elements. AltiVec defines a set of instructions to manipulate this vectordataeffectivelyinparallelwithcoreCPUinstructionprocessing.AltiVecoperationsinclude suchcomputationsassum-across,multiply-sum,simultaneousdatadistribute(store),anddatagather (load)instructions. Programmers have used the AltiVec hardware to enable very fast software computations commonly found in signal-processing and network elements. Examples include fast Fourier Transform, digital signal processing such as filtering, MPEG video coding and encoding, and fast generationofencryptionprotocolssuchasDES,MD5,andSHA1. OtherchipsintheFreescalelineupofstand-aloneprocessorsincludetheMPC7410,MPC7445, MPC7447,MPC745x,andMPC7xxfamily. 3.1.4.CompanionChipsets Stand-alone processors such as those just described require support logic to connect to and enableexternalperipheraldevicessuchasmainsystemmemory(DRAM),ROMorFlashmemory, system busses such as PCI, and other peripherals, such as keyboard controllers, serial ports, IDE interfaces,andthelike.Thissupportlogicisoftenaccomplishedbycompanionchipsets,whichmay evenbepurpose-designedspecificallyforafamilyofprocessors. Forexample,thePentiumMissupportedbyonesuchchipset,calledthe855GM.The855GM chipset is the primary interface to graphics and memorythus, the suffix-GM. The 855GM has been optimized as a companion to the Pentium M. Figure 3-1 illustrates the relationship between the processorandchipsetsinthistypeofhardwaredesign. Figure3-1.Processor/chipsetrelationship Notetheterminologythathasbecomecommonfordescribingthesechipsets.TheIntel855GMis anexampleofwhatiscommonlyreferredtoasanorthbridgechipbecauseitisdirectlyconnectedto theprocessor'shigh-speedfrontsidebus(FSB).AnothercompanionchipthatprovidesI/OandPCI bus connectivity is similarly referred to as the southbridge chip because of its position in the architecture. The southbridge chip (actually, an I/O controller) in these hardware architectures is responsible for providing interfaces such as those shown in Figure 3-1, including Ethernet, USB, IDE,audio,keyboard,andmousecontrollers. OnthePowerPCside,theTundraTsi110HostBridgeforPowerPCisanexampleofachipset thatsupportsthestand-alonePowerPCprocessors.TheTsi110supportsseveralinterfacefunctions for many common stand-alone PowerPC processors. The Tundra chip supports the Freescale MPC74xx and the IBM PPC 750xx family of processors. The Tundra chip can be used by these processorstoprovidedirectinterfacestothefollowingperipherals: •DDRDRAM,integratedmemorycontroller •Ethernet(theTundraprovidesfourgigabitEthernetports) •PCIExpress(supports2PCIExpressports) •PCI/X(PCI2.3,PCI-X,andCompactPCI[cPCI]) •Serialports •I2C •Programmableinterruptcontroller •Parallelport Many manufacturers of chipsets exist, including VIA Technologies, Marvell, Tundra, nVidia, Intel, and others. Marvell and Tundra primarily serve the PowerPC market, whereas the others specializeinIntelarchitectures.Hardwaredesignsbasedononeofthemanystand-aloneprocessors, such as Intel x86, IBM, or Freescale PowerPC, need to have a companion chipset to interface to systemdevices. One of the advantages of Linux as an embedded OS is rapid support of new chipsets. Linux currently has support for those chipsets mentioned here, as well as many others. Consult the Linux sourcecodeandconfigurationutilityforinformationonyourchosenchipset. 3.2.IntegratedProcessors:SystemsonChip Intheprevioussection,wehighlightedstand-aloneprocessors.Althoughtheyareusedformany applications, including some high-horsepower processing engines, the vast majority of embedded systemsemploysometypeofintegratedprocessor,orsystemonchip(SOC).Literallyscores,ifnot hundreds,existtochoosefrom.Weexamineafewfromtheindustryleadersandlookatsomeofthe featuresthatseteachgroupapart.Asinthesectiononstand-aloneprocessors,wefocusonlyonthose integratedprocessorswithstrongLinuxsupport. Several major processor architectures exist, and each architecture has examples of integrated SOCs.PowerPChasbeenatraditionalleaderinmanynetworking-andtelecommunications-related embedded applications, while MIPS might have the market lead in lower-end consumer-grade equipment.[19] ARM is used in many cellular phones. These represent the major architectures in widespread use in embedded Linux systems. However, as you will see in Chapter 4, "The Linux Kernel: A Different Perspective," Linux supports more than 20 different hardware architectures today. 3.2.1.PowerPC PowerPC is a Reduced Instruction Set Computer (RISC) architecture jointly designed by engineersfromApple,IBM,andMotorola'ssemiconductordivision(nowanindependententityspun offasFreescaleSemiconductor).ManygooddocumentsdescribethePowerPCarchitectureingreat detail. Consult the " Suggestions for Additional Reading " at the end of this chapter as a starting point. PowerPCprocessorshavefoundtheirwayintoembeddedproductsofeverydescription.From automotive, consumer, and networking applications to the largest data and telecommunications switches,PowerPCisoneofthemostpopulararchitecturesforembeddedapplications.Becauseof this popularity, there exists a large array of hardware and software solutions from numerous manufacturerstargetedatPowerPC. 3.2.2.AMCCPowerPC Some of the examples later in this book are based on the AMCC PowerPC 440EP Embedded Processor. The 440EP is a popular integrated processor found in many networking and communicationsproducts.Thefollowinglisthighlightssomeofthefeaturesofthe440EP: •On-chipdual-data-rate(DDR)SDRAMcontroller •IntegratedNANDFlashcontroller •PCIbusinterface •Dual10/100MbpsEthernetports •On-chipUSB2.0interface •Uptofouruser-configurableserialports •DualI2Ccontrollers •ProgrammableInterruptController •SerialPeripheralInterface(SPI)controller •Programmabletimers •JTAGinterfacefordebugging Thisisindeed acomplete systemonchip(SOC).Figure3-2isablockdiagram oftheAMCC PowerPC 440EP Embedded Processor. With the addition of memory chips and physical I/O hardware,acompletehigh-endembeddedsystemcanbebuiltaroundthisintegratedmicroprocessor withminimalinterfacecircuitryrequired. Figure3-2.AMCCPPC440EPEmbeddedProcessor(CourtesyAMCCCorporation) Many manufacturers offer reference hardware platforms to enable a developer to explore the capabilities of the processor or other hardware. The examples later in this book (Chapters 14, "KernelDebuggingTechniques";and15,"DebuggingEmbeddedLinuxApplications")wereexecuted on the AMCC Yosemite board, which is the company's reference platform containing the 440EP showninFigure3-2. Numerous product configurations are available with PowerPC processors. As demonstrated in Figure3-2,theAMCC440EPcontainssufficientI/Ointerfacesformanycommonproducts,withvery littleadditionalcircuitry.Becausethisprocessorcontainsanintegratedfloating-pointunit(FPU),it isideallysuitedforproductssuchasnetwork-attachedimagingsystems,generalindustrialcontrol, andnetworkingequipment. AMCC'sPowerPCproductlineupincludesseveralconfigurationspoweredbytwoprovencores. Their 405 core products are available in configurations with and without Ethernet controllers. All 405coreconfigurationsincludeintegratedSDRAMcontrollers,dualUARTsforserialports,I 2C for low-level onboard management communications, general-purpose I/O pins, and integral timers. TheAMCC405coreintegratedprocessorsprovideeconomicalperformanceonaprovencorefora widerangeofapplicationsthatdonotrequireahardwareFPU. The AMCC 440-based core products raise the performance level and add peripherals. The 440EP featured in some of our examples includes a hardware FPU. The 440GX adds two triplespeed10/100/1000MB Ethernetinterfaces(inadditiontothe two10/100MbpsEthernetports)and TCP/IP hardware acceleration for high-performance networking applications. The 440SP adds hardwareaccelerationforRAID5/6applications.AlltheseprocessorshavematureLinuxsupport. Table3-1summarizesthehighlightsoftheAMCC405xxfamily. Table3-1.AMCCPowerPC405xxHighlightsSummary Feature 405CR 405EP 405GP 405GPr PowerPC405133- PowerPC405133- PowerPC405133- PowerPC405266Core/speeds 266MHz 333MHz 266MHz 400MHz DRAM SDRAM/133 SDRAM/133 SDRAM/133 SDRAM/133 controller Ethernet N 2 1 1 10/100 GPIOlines 23 32 24 24 UARTs 2 2 2 2 DMA 4channel 4channel 4channel 4channel controller I2Ccontroller Y Y Y Y PCIhost N Y Y Y controller Interrupt Y Y Y Y controller SeetheAMCCwebsite,atwww.amcc.com/embedded,forcompletedetails. Table3-2summarizesthefeaturesoftheAMCC440xxfamilyofprocessors. Table3-2.AMCCPowerPC440xxHighlightsSummary Feature 440EP 440GP 440GX 440SP PowerPC440333- PowerPC440400- PowerPC440533- PowerPC440533Core/speeds 667MHz 500MHz 800MHz 667MHz DRAM DDR DDR DDR DDR controller Ethernet 2 2 2 viaGigE 10/100 Gigabit Ethernet N GPIOlines 64 UARTs 4 DMA 4channel controller I2Ccontroller 2 PCIhost Y controller SPIcontroller Y Interrupt Y controller N 2 1 32 2 32 2 32 3 4channel 4channel 3channel 2 2 2 PCI-X PCI-X threePCI-X N N N Y Y Y 3.2.3.FreescalePowerPC FreescaleSemiconductorhasalargerangeofPowerPCprocessorswithintegratedperipherals. The manufacturer is currently advertising its PowerPC product portfolio centered on three broad vertical-market segments: networking, automotive, and industrial. Freescale PowerPC processors have enjoyed enormous success in the networking market segment. This lineup of processors has wideappealinalargevarietyofnetworkequipment,fromthelowendtothehighendoftheproduct space. Inarecentpressrelease,FreescaleSemiconductorannouncedthatithadshippedmorethan200 millionintegratedcommunicationsprocessors.[20]Partofthissuccessisbasedaroundthecompany's PowerQUICCproductline.ThePowerQUICCarchitecturehasbeenshippingformorethanadecade. It is based on a PowerPC core integrated with a QUICC engine (also called a communications processormoduleorCPMintheFreescaleliterature).TheQUICCengineisanindependentRISC processor designed to offload the communications processing from the main PowerPC core, thus freeingupthePowerPCcoretofocusoncontrolandmanagementapplications.TheQUICCengineis acomplexbuthighlyflexiblecommunicationsperipheralcontroller. [5]Initscurrentincarnation,PowerQUICCencompassesfourgeneralfamilies.Forconvenience, aswediscussthesePowerQUICCproducts,werefertoitasPQ. The PQ I family includes the original PowerPC-based PowerQUICC implementations and consistsoftheMPC8xxfamilyofprocessors.Theseintegratedcommunicationsprocessorsoperateat 50-133MHzandfeaturetheembeddedPowerPC8xxcore.ThePQIfamilyhasbeenusedforATM andEthernetedgedevicessuchasroutersforthehomeandsmalloffice(SOHO)market,residential gateways,ASDLandcablemodems,andsimilarapplications. TheCPMorQUICCengineincorporatestwouniqueandpowerfulcommunicationscontrollers. TheSerialCommunication Controller(SCC)isaflexibleserialinterfacecapableofimplementing many serial-based communications protocols, including Ethernet, HDLC/SDLC, AppleTalk, synchronousandasynchronousUARTs,IrDA,andotherbitstreamdata. TheSerialManagementController(SMC)isamodulecapableofsimilarserial-communications protocols,andincludessupportforISDN,serialUART,andSPIprotocols. Using a combination of these SCCs and SMCs, it is possible to create very flexible I/O combinations. An internal time-division multiplexer even allows these interfaces to implement channelizedcommunicationssuchasT1andE1I/O. Table3-3summarizesasmallsamplingofthePQIproductline. Table3-3.FreescaleSelectPowerQUICCIHighlights Feature MPC850 MPC860 MPC875 MPC885 PowerPC8xx PowerPC8xx PowerPC8xx PowerPC8xx Core/speeds Upto80MHz Upto80MHz Upto133MHz Upto133MHz DRAMcontroller Y Y Y Y USB Y N Y Y SPIcontroller Y Y Y Y I2Ccontroller Y Y Y Y SCCcontrollers 2 SMCcontrollers 2 Securityengine N DedicatedFastEthernetcontroller N 4 2 N N 1 1 Y 2 3 1 Y 2 ThenextstepupintheFreescalePowerPCproductlineisPowerQUICCII.PQIIincorporates thecompany'sG2PowerPCcorederivedfromthe603eembeddedPowerPCcore.Theseintegrated communications processors operate at 133-450MHz and feature multiple 10/100Mbps Ethernet interfaces,securityengines,andATMandPCIsupport,amongmanyothers.ThePQIIencompasses theMPC82xxproducts. PQ II adds two new types of controllers to the QUICC engine. The FCC is a full-duplex fast serial communications controller. The FCC supports high-speed communications such as 100Mbps EthernetandT3/E3upto45Mbps.TheMCCisamultichannelcontrollercapableof128KBx64KB channelizeddata. Table3-4summarizesthehighlightsofselectedPowerQUICCIIprocessors. Table3-4.FreescaleSelectPowerQUICCIIHighlights Feature MPC8250 MPC8260 MPC8272 MPC8280 G2/603e150G2/603e100G2/603e266G2/603e266Core/speeds 200MHz 300MHz 400MHz 400MHz DRAM Y Y Y Y controller USB N N Y ViaSCC4 SPIcontroller Y Y Y Y I2Ccontroller Y Y Y Y SCC 4 4 3 4 controllers SMC 2 2 2 2 controllers FCC 3 3 2 3 controllers MCC 1 2 0 2 controllers Based on the Freescale PowerPC e300 core (evolved from the G2/603e), the PowerQUICC II Profamilyoperatesat266-667MHzandfeaturessupportforGigabitEthernet,dualdatarate(DDR) SDRAMcontrollers,PCI,high-speedUSB,securityacceleration,andmore.ThesearetheMPC83xx familyofprocessors.ThePQIIandPQIIProfamiliesofprocessorshavebeendesignedintoawide varietyofequipment,suchasLANandWANswitches,hubsandgateways,PBXsystems,andmany othersystemswithsimilarcomplexityandperformancerequirements. ThePowerQUICCIIProcontainsthreefamilymemberswithouttheQUICCengine,andtwothat arebasedonanupdatedversionoftheQUICCengine.TheMPC8358EandMPC8360Ebothadda newUniversalCommunicationsController,whichsupportsavarietyofprotocols. Table3-5summarizesthehighlightsofselectmembersofthePQIIProfamily. Table3-5.FreescaleSelectPowerQUICCIIProHighlights Feature MPC8343E MPC8347E MPC8349E MPC8360E Core/speeds e300266-400MHz e300266-667MHz e300400-667MHz e300266-667MHz DRAMcontroller Y-DDR Y-DDR Y-DDR Y-DDR USB Y 2 2 Y SPIcontroller Y Y Y Y I2Ccontroller 2 2 2 2 Ethernet10/100/1000 2 2 2 ViaUCC UART 2 2 2 2 PCIcontroller Y Y Y Y Securityengine Y Y Y Y MCC 0 0 0 1 UCC 0 0 0 8 AtthetopofthePowerQUICCfamilyarethePQIIIprocessors.Theseoperatebetween600MHz and1.5GHz.Theyarebasedonthee500coreandsupportGigabitEthernet,DDRSDRAM,RapidIO, PCIandPCI/X,ATM,HDLC,andmore.ThisfamilyincorporatestheMPC85xxproductline.These processors have found their way into high-end products such as wireless base station controllers, opticaledgeswitches,centralofficeswitches,andsimilarequipment. Table3-6highlightssomeofthePQIIIfamilymembers. Table3-6.FreescaleSelectPowerQUICCIIIHighlights Feature MPC8540 MPC8548E MPC8555E MPC8560 Core/speeds e500Upto1.0GHz e500Upto1.5GHz e500Upto1.0GHz e500Upto1.0GHz DRAMcontroller Y-DDR Y-DDR Y-DDR Y-DDR USB N N ViaSCC N SPIcontroller N N Y Y I2Ccontroller Y Y Y Y Ethernet10/100 1 ViaGigE ViaSCC ViaSCC GigabitEthernet 2 4 2 2 UART 2 2 2 ViaSCC PCIcontroller PCI/PCI-X PCI/PCI-X PCI PCI/PCI-X RapidIO Y Y N Y Securityengine N Y Y N SCC 3 4 FCC 2 3 SMC 2 0 MCC 0 2 3.2.4.MIPS Youmightbesurprisedtolearnthat32-bitprocessorsbasedontheMIPSarchitecturehavebeen shipping for more than 20 years. The MIPS architecture was designed in 1981 by a Stanford UniversityengineeringteamledbyDr.JohnHennessey,wholaterwentontoformMIPSComputer Systems, Inc. That company has morphed into the present-day MIPS Technologies, whose primary roleisthedesignandsubsequentlicensingofMIPSarchitectureandcores. The MIPS core has been licensed by many companies, several of which have become powerhouses in the embedded processor market. MIPS is a Reduced Instruction Set Computing (RISC)architecturewithboth32-bitand64-bitimplementationsshippinginmanypopularproducts. MIPSprocessorsarefoundinalargevarietyofproducts,fromhigh-endtoconsumerdevices.Itis publicknowledgethatMIPSprocessorspowermanypopularwell-knownconsumerproducts,such as Sony high definition television sets, Linksys wireless access points, and the popular Sony PlayStation2gameconsole.[21] The MIPS Technology website lists 73 licensees who are currently engaged in manufacturing productsusingMIPSprocessorcores.Someofthesecompaniesarehouseholdnames,aswithSony, Texas Instruments, Cisco's Scientific Atlanta (a leading manufacturer of cable TV set-top boxes), Motorola, and others. Certainly, one of the largest and most successful of these is Broadcom Corporation. 3.2.5.BroadcomMIPS Broadcom is a leading supplier of SOC solutions for markets such as cable TV set-top boxes, cablemodems, HDTV,wirelessnetworks,GigabitEthernet,andVoiceoverIP(VoIP).Broadcom's SOCshavebeenverypopularinthesemarkets.WementionedearlierthatyoulikelyhaveLinuxin yourhomeevenifyoudon'tknowit.Chancesare,ifyoudo,itisrunningonaBroadcomMIPS-based SOC. In 2000, Broadcom acquired SiByte Inc., which resulted in the communications processor product lineup the company is currently marketing. These processors currently ship in single-core, dual-core,andquad-coreconfigurations.ThecompanystillreferstothemasSiByteprocessors. Thesingle-coreSiByteprocessorsincludetheBCM1122andBCM1125H.Theyarebothbased ontheMIPS64coreandoperateatclockspeedsat400-900MHz.Theyincludeon-chipperipheral controllers such as DDR SDRAM controller, 10/100Mbps Ethernet, and PCI host controller. Both includeSMBusserialconfigurationinterface,PCMCIA,andtwoUARTsforserialportconnections. The BCM1125H includes a triple-speed 10/100/1000Mbps Ethernet controller. One of the more strikingfeaturesoftheseprocessorsistheirpowerdissipation.Bothfeaturea4Woperatingbudgetat 400MHzoperation. Thedual-coreSiByteprocessorsincludetheBCM1250,BCM1255,andBCM1280.Alsobased ontheMIPS64core,theseprocessorsoperateatclockratesfrom600MHz(BCM1250)toashighas 1.2GHz(BCM1255andBCM1280).Thesedual-corechipsincludeintegratedperipheralcontrollers suchasDDRSDRAMcontrollers,variouscombinationsofGigabitEthernetcontrollers,64-bitPCIX interfaces,andSMBus,PCMCIA,andmultipleUARTinterfaces.Liketheirsingle-corecousins, these dual-core implementations also feature low power dissipation. For example, the BCM1255 featuresa13Wpowerbudgetat1GHzoperation. The quad-core SiByte processors include the BCM1455 and BCM1480 communications processors.AswiththeotherSiByteprocessors,thesearebasedontheMIPS64core.Thecorescan be run from 800MHz to 1.2GHz. These SOCs include integrated DDR SDRAM controllers, four separate Gigabit Ethernet MAC controllers, and 64-bit PCI-X host controllers, and also contain SMBus,PCMCIA,andfourserialUARTs. Table3-7summarizesselectBroadcomSiByteprocessors. Table3-7.BroadcomSelectSiByteProcessorHighlights Feature BCM1125H BCM1250 BCM1280 BCM1480 SB-1 DualSB-1 DualSB-1 QuadSB-1 Core/speeds MIPS64 MIPS64 MIPS64 MIPS64 400-900MHz 600-1000MHz 800-1200MHz 800-1200MHz DRAMcontroller Y-DDR Y-DDR Y-DDR Y-DDR Serialinterface 2-55Mbps 2-55Mbps 4UART 4UART SMBusinterface 2 2 2 2 PCMCIA Y Y Y Y GigabitEthernet(10/100/1000Mbps) 2 3 4 4 PCIcontroller Y Y YPCI/PCI-X YPCI/PCI-X Securityengine High-speedI/O(HyperTransport) N 1 N 1 N 3 3 3.2.6.AMDMIPS AdvancedMicroDevicesalsoplaysasignificantroleintheembeddedMIPScontrollermarket. The company's 2002 acquisition of Alchemy Semiconductor garnered several popular single-chip integratedSOCsbasedontheMIPS32coreandarchitecture.TheAlchemylinefromAMDisbased on the popular MIPS32 core. All feature relatively low power dissipation and a high level of onboardsystemintegration. TheAu1000andAu1100operateatclockratesof266-500MHz.BothfeatureonboardSDRAM controllers and separate bus controllers for attachment to external devices such as Flash and PCMCIA.Table3-8summarizesthecurrentAlchemyproductline. Table3-8.AMDAlchemyMIPSHighlightsSummary Feature[22] Au1000 Au1100 Au1200 Au1500 Au1550 MIPS32266MIPS32333MIPS32333MIPS32333MIPS32333Core/speeds 500MHz 500MHz 500MHz 500MHz 500MHz DRAM SDRAM SDRAM DDRSDRAM SDRAM DDRSDRAM controller Ethernet 2 1 2 2 10/100 GPIOlines 32 48 48 39 43 UARTs 4 3 2 2 3 USB1.1 Host+device Host+device USB2.0 Host+device Host+device AC-97audio 1 1 ViaSPC 1 ViaSPC I2S 1 1 ViaSPC ViaSPC controller SD/MMC N 2 2 N N 3.2.7.OtherMIPS As we pointed out earlier, nearly 100 current MIPS licensees are shown on the MIPS Technologies licensees web page, at www.mips.com/content/Licensees/ProductCatalog/licensees. Unfortunately, it is not possible in the space provided here to cover them all. Start your search at the MIPS technologieswebsiteforagoodcross-sectionoftheMIPSprocessorvendors. Forexample,ATITechnologiesusesaMIPScoreinitsXilleonset-topboxfamilyofchipsets. Cavium Network's Octeon family uses MIPS64 cores in a variety of multicore processor implementations. Integrated Device Technology, Inc., (IDT) has a family of integrated communications processors called Interprise, based on the MIPS architecture. PMC-Sierra, NEC, Toshiba, and others have integrated processors based on MIPS. All of these and more are well supportedunderLinux. 3.2.8.ARM The ARM architecture has achieved a very large market share in the consumer electronics marketplace. Many popular and now ubiquitous products contain ARM cores. Some well-known examples include the Sony PlayStation Portable (PSP), Apple iPod Nano,[23] Nintendo Game Boy Micro and DS, TomTom GO 300 GPS, and the Motorola E680i Mobile Phone, which features embeddedLinux.ProcessorscontainingARMcorespoweramajorityoftheworld'sdigitalcellular phones,accordingtotheARMCorporateBackgrounderatwww.arm.com/miscPDFs/3822.pdf. The ARM architecture is developed by ARM Holdings, plc and licensed to semiconductor manufacturersaroundtheglobe.Manyoftheworld'sleadingsemiconductorcompanieshavelicensed ARMtechnologyandarecurrentlyshippingintegratedprocessorsbasedononeoftheseveralARM cores. 3.2.9.TIARM Texas Instruments uses ARM cores in the OMAP family of integrated processors. These processors contain many integrated peripherals intended to be used as single-chip solutions for various consumer products, such as cellular handsets, PDAs, and similar multimedia platforms. In addition to the interfaces commonly found on integrated processors, such as UARTs and I2C, the OMAPdevicescontainawiderangeofspecial-purposeinterfaces,includingthefollowing: •LCDscreenandbacklightcontrollers •Buzzerdriver •Camerainterface •MMC/SDcardcontroller •Battery-managementhardware •USBclient/hostinterfaces •Radiomodeminterfacelogic •Integrated2Dor3Dgraphicsaccelerators •Integratedsecurityaccelerator •S-Videooutputs •IrDAcontroller •DACsfordirectTV(PAL/NTSC)videooutput •IntegratedDSPsforvideoandaudioprocessing Many popular cellular handsets and PDA devices have been marketed based on the TI OMAP platform.BecausetheyarebasedonanARMcore,theseprocessorsaresupportedbyLinuxtoday. Table3-9comparessomeofthemorerecentmembersoftheTIOMAPfamily. Table3-9.TIARMOMAPHighlightsSummary Feature OMAP1710 OMAP2420 OMAP2430 OMAP3430 ARM926 ARM11 ARM1136 ARMCortexA8 TEJ Core/speeds Upto 330MHz 330MHz 550MHz 200MHz DRAMcontroller Y Y Y Y UARTs Y Y Y Y USB Client+host Client+host Client+host Client+host I2Ccontroller Y Y Y Y MMC-SDinterface Y Y Y Y Keypadcontroller Y Y Y Y Camerainterface Y Y Y Y Graphicsaccelerator 2D 2D/3D 2D/3D Y IntegratedDSP TM320C55x TM320C55x N N ImagingVideo Videoacceleration ImagingVideo ImagingVideo N Accelerator(IVA2 hardware Accelerator(IVA) Accelerator(IVA2) +) Securityaccelerator Audiocodecsupport Bluetooth&RFmodem supportinterface LCDcontroller Y Y Y Y Y Y Y Y Y Y Y Y Y Displaycontrollers N Y PAL/NTSC VGA/QVGA Y PAL/NTSC VGA/QVGA Y PAL/NTSC QVGA/XGA 3.2.10.FreescaleARM ThesuccessoftheARMarchitectureismademoreevidentbythefactthatleadingmanufacturers of competing architectures have licensed ARM technology. As a prime example, Freescale Semiconductor has licensed ARM technology for its line of i.MX application processors. These popularARM-basedintegratedprocessorshaveachievedwidespreadindustrysuccessinmultimedia consumerdevicessuchasportablegameplatforms,PDAs,andcellularhandsets. TheFreescaleARMproductportfolioincludesthei.MX21andi.MX31applicationprocessors. Thei.MX21featuresanARM9core,andthei.MX31hasanARM11core.LiketheirTIcounterparts, theseSOCscontainmanyintegratedperipheralsrequiredbyportableconsumerelectronicsdevices withmultimediarequirements.Thei.MX21/31containsomeofthefollowingintegratedinterfaces: •Graphicsaccelerator •MPEG-4encoder •KeypadandLCDcontrollers •Camerainterface •Audiomultiplexer •IrDAinfraredI/O •SD/MMCinterface •NumerousexternalI/O,suchasPCMCIA,USB,DRAMcontrollers,andUARTsforserialport connection 3.2.11.IntelARMXScale Intel manufactures and markets several integrated processors based on the ARM v5TE architecture.IntelusestheXScalenameforthearchitecture.Theseproductsaregroupedintoseveral applicationcategories.Table3-10summarizestheXScalefamiliesbyapplicationtype. Table3-10.IntelXScaleProcessorSummary Category Application ExampleProcessors Application CellularhandsetsandPDAs PXA27x,PXA29x processors High-speeddataprocessingusedinstorage,printing, I/Oprocessors IOP331/332/333 telematics,andsoon Network Networkingandcommunicationsdataplaneprocessing, IXP425,IXP465 processors fastpacketprocessing,andsoon IXP2350,IXP2855 Many consumer and networking products have been developed using Intel XScale architecture processors. Some well-known examples include the GPS iQue M5 from Garmin, the iPAQ by Hewlett-Packard, smart phones from Palm (Treo) and Motorola (A760), and many others. Linux currentlysupportsalltheseprocessors. Intel's network processors are found in high-performance networking equipment where requirements exist for fast data-path processing. Examples include deep packet inspection, data encryption/decryption, packet filtering, and signal processing. These network processors each contain an ARM core coupled with one or more dedicated processing engines, called a network processingengine(NPE).TheseNPEsarededicatedtospecificdata-pathmanipulationinrealtime atwirespeeds.TheNPEisamicroprocessor,inthesensethatitcanexecutemicrocodedalgorithms inparallelwiththethreadofexecutionintheARMcore.RefertotheIntelwebsite,at www.intel.com, foradditionalinformationonthispowerfulfamilyofintegratedprocessors. 3.2.12.OtherARM More than 100 semiconductor companies are developing integrated solutions based on ARM technologyfartoomanytolisthere.Manyofferspecializedapplicationprocessorsservingvertical marketssuchasthehandsetmarket,storageareanetworking,networkprocessing,andtheautomotive market,aswellasmanymore.ThesecompaniesincludeAltera,PMC-Sierra,SamsungElectronics, PhilipsSemiconductor,Fujitsu,andmore.SeetheARMTechnologieswebsiteatwww.arm.comfor additionalARMlicenseesandinformation. 3.2.13.OtherArchitectures We have covered the major architectures in widespread use in embedded Linux systems. However,forcompleteness,youshouldbeawareofotherarchitecturesforwhichsupportexistsin Linux.ArecentLinuxsnapshotrevealed25architecturebranches(subdirectories).Insomeinstances, the64-bitimplementationofanarchitectureisseparatedfromits32-bitcounterpart.Inothercases, portsarenotcurrentorarenolongermaintained. TheLinuxsourcetreecontainsportsforSunSparcandSparc64,theXtensafromTensilica,and thev850fromNEC,tonameafew.Spendafewminuteslookingthroughthearchitecturebranchesof theLinuxkerneltoseetherangeofarchitecturesforwhichLinuxhasbeenported.Beware,however, that not all these architectures might be up-to-date in any given snapshot. You can be reasonably certainthatthemajorarchitecturesarefairlycurrent,buttheonlywaytobecertainistofollowthe developmentintheLinuxcommunityorconsultwithyourfavoriteembeddedLinuxvendor.Appendix E, "Open Source Resources," contains a list of resources you can consult to help you stay current withLinuxdevelopments. 3.3.HardwarePlatforms The idea of a common hardware reference platform is not new. The venerable PC/104 and VMEbus are two examples of hardware platforms that have withstood the test of time in the embeddedmarket.[24]MorerecentsuccessfulplatformsincludeCompactPCIanditsderivatives. 3.3.1.CompactPCI The CompactPCI (cPCI) hardware platform is based on PCI electrical standards and Eurocard physicalspecifications.cPCIhasthefollowinggeneralfeatures: •Verticalcardsof3Uor6Uheights •Latchsystemforsecuringandejectingcards •Front-orrear-panelI/Oconnectionssupported •High-densitybackplaneconnector •Staggeredpowerpinsforhot-swapsupport •Supportbymanyvendors •CompatibilitywithstandardPCIchipsets You can view highlights of and obtain specifications for the cPCI architecture at the PCI Industrial Computer Manufacturers Group (PICMG) cPCI web page, at www.picmg.org/compactpci.stm. 3.3.2.ATCA AsuccessortothesuccessfulcPCI,AdvancedTelecommunicationsComputingArchitectureisthe name given to the architecture and platforms designed around the PICMG 3. x series of specifications.Manytop-tierhardwaremanufacturersareshippingordevelopingnewATCA-based platforms. The primary applications for ATCA platforms are carrier-class telecommunications switchingandtransportequipment,andhigh-enddata-centerserverandstorageequipment. ATCA platforms are leading the industry trend away from in-house proprietary hardware and software platforms. Many of the largest equipment manufacturers in the telecommunications and networking markets have been slowly moving away from the custom, in-house-designed hardware platforms.Thistrendisalsoevidentinthesoftwareplatforms,fromoperatingsystemstoso-called middleware such as high-availability and protocol stack solutions. Downsizing and time-to-market pressuresaretwokeyfactorsdrivingthistrend. ATCAisdefinedbyseveralPICMGspecifications.Table3-11summarizesthesespecifications. Table3-11.ATCAPICMG3.xSpecificationSummary Specification Summary Mechanicalspecifications,includinginterconnects,power,cooling,andbasesystem PICMG3.0 management PICMG3.1 EthernetandFiberChannelswitchingfabricinterface PICMG3.2 Infinibandswitchingfabricinterface PICMG3.3 StarFabricinterface PICMG3.4 PCIExpressinterface PICMG3.5 RapidIOInterface The platforms described in this section are the most relevant in any discussion of embedded Linux platforms today. Especially with ATCA, the industry is increasingly moving toward commercial off-the-shelf (COTS) technology. Both ATCA and Linux play increasingly important rolesinthisindustrytrend. 3.4.ChapterSummary •Manystand-aloneprocessorsaresupportedunderLinux.Themostwidelysupportedofthese areIA32/IA64andPowerPCarchitectures.Thesestand-aloneprocessorsareusedasbuildingblocks tobuildvery-high-performancecomputingengines.WepresentedseveralexamplesfromIntel,IBM, andFreescale. • Integrated processors, or systems on chip (SOCs), dominate the embedded Linux landscape. ManyvendorsandseveralpopulararchitecturesareusedinembeddedLinuxdesigns.Severalofthe mostpopulararepresentedinthischapterbyarchitectureandmanufacturer. • An increasingly popular trend is to move away from proprietary hardware and software platforms,towardcommercialoff-the-shelf(COTS)solutions.Twopopularplatformsinwidespread useinembeddedLinuxsystems:cPCIandATCA. 3.4.1.SuggestionsForAdditionalReading PowerPC32-bitarchitecturereferencemanual: ProgrammingEnvironmentsManualfor32-BitImplementationsofthePowerPC ArchitectureRevision2 FreescaleSemiconductor,Inc. www.freescale.com/files/product/doc/MPCFPE32B.pdf PowerPC64-bitarchitecturereference: TheProgrammingEnvironmentsManualfor64-BitMicroprocessorsVersion3.0 InternationalBusinessMachines,Inc. ShortsummaryofPowerPCarchitecture: ADeveloper'sGuidetothePOWERArchitecture BrettOlsson,ProcessorArchitect,IBMCorp. AnthonyMarsala,SoftwareEngineer,IBMCorp. www-128.ibm.com/developerworks/linux/library/l-powarch/ IntelXScalesummarypage www.intel.com/design/intelxscale/ Chapter4.TheLinuxKernelADifferentPerspective Ifyouwanttolearnaboutkernelinternals,manygoodbooksareavailableonkerneldesignand operation.SeveralarepresentedinSection4.5.1,"SuggestionsforAdditionalReading,"inthisand other chapters throughout the book. However, very little has been written about how the kernel is organizedandstructuredfromaprojectperspective.Whatifyou'relookingfortherightplacetoadd somecustomsupportforyournewembeddedproject?Howdoyouknowwhichfilesareimportant foryourarchitecture? Atfirstglance,itmightseemanalmostimpossibletasktounderstandtheLinuxkernelandhowto configureitforaspecificplatformorapplication.InarecentLinuxkernelsnapshot,theLinuxkernel sourcetreeconsistsofmorethan20,000filesthatcontainmorethansixmillionlinesandthat'sjust thebeginning.Youstillneedtools,arootfilesystem,andmanyLinuxapplicationstomakeausable system. This chapter introduces the Linux kernel and covers how the kernel is organized and how the sourcetreeisstructured.Wethenexaminethecomponentsthatmakeupthekernelimageanddiscuss thekernelsourcetreelayout.Followingthis,wepresentthedetailsofthekernelbuildsystemandthe filesthatdrivethekernelconfigurationandbuildsystem.Thischapterconcludesbyexaminingwhat isrequiredforacompleteembeddedLinuxsystem. 4.1.Background Linus TorvaldswrotetheoriginalversionofLinuxwhilehewasastudentattheUniversityof HelsinkiinFinland.Hisworkbeganin1991.InAugustofthatyear,Linuspostedthisnow-famous announcementoncomp.os.minix: From:[email protected](LinusBenedictTorvalds) Newsgroups:comp.os.minix Subject:Whatwouldyouliketoseemostinminix? Summary:smallpollformynewoperatingsystem Message-ID:<[email protected]> Date:25Aug9120:57:08GMT Organization:UniversityofHelsinki HelloeverybodyoutthereusingminixI'mdoinga(free)operatingsystem(justahobby,won'tbebigandprofessional likegnu)for386(486)ATclones.Thishasbeenbrewingsinceapril,andis startingtogetready.I'dlikeanyfeedbackonthingspeoplelike/dislikein minix,asmyOSresemblesitsomewhat(samephysicallayoutofthefile-system (duetopracticalreasons)amongotherthings). I'vecurrentlyportedbash(1.08)andgcc(1.40),andthingsseemtowork.This impliesthatI'llgetsomethingpracticalwithinafewmonths,andI'd liketo knowwhatfeaturesmostpeoplewouldwant.Anysuggestionsarewelcome,butI won'tpromiseI'llimplementthem:-) Linus([email protected]) PS.Yes-it'sfreeofanyminixcode,andithasamulti-threadedfs. ItisNOTprotable(uses386taskswitchingetc),anditprobablynever willsupportanythingotherthanAT-harddisks,asthat'sallIhave:-(. Sincethatinitialrelease,Linuxhasmaturedintoafull-featuredoperatingsystemwithrobustness, reliability,andhigh-endfeaturesthatrivalthoseofthebestcommercialoperatingsystems.Bysome estimates,morethanhalfoftheInternetserversontheWebarepoweredbyLinuxservers.Itisno secret that the online search giant Google uses a large collection of low-cost PCs running a faulttolerantversionofLinuxtoimplementitspopularsearchengine. 4.1.1.KernelVersions YoucanobtainthesourcecodeforaLinuxkernelandcomplementarycomponentsinnumerous places.YourlocalbookstoremighthaveseveralversionsascompanionCD-ROMsinbooksabout Linux.YoualsocandownloadthekernelitselforevencompleteLinuxdistributionsfromnumerous locationsontheInternet.TheofficialhomefortheLinuxkernelisfoundatwww.kernel.org.Youwill often hear the terms mainline source or mainline kernel referring to the source trees found at kernel.org. Asthisbookisbeingwritten,LinuxVersion2.6isthecurrentversion.Earlyinthedevelopment cycle,thedeveloperschoseanumberingsystemdesignedtodifferentiatebetweenkernelsourcetrees intended for development and experimentation and source trees intended to be stable, productionreadykernels.Thenumberingschemecontainsamajorversionnumber,aminorversionnumber,and thenasequencenumber.BeforeLinuxVersion2.6,iftheminorversionnumberiseven,itdenotesa productionkernel;ifitisodd,itdenotesadevelopmentkernel.Forexample: •Linux2.4.xProductionkernel •Linux2.5.xExperimental(development) •Linux2.6.xProductionkernel Currently, there is no separate development branch of the Linux 2.6 kernel. All new features, enhancements, and bug fixes are funneled through a series of gatekeepers who ultimately filter and push changes up to the top-level Linux source trees maintained by Andrew Morton and Linus Torvalds. Itiseasytotellwhatkernelversionyouareworkingwith.Thefirstfewlinesofthetop-level makefile[25]inakernelsourcetreedetailtheexactkernelversionrepresentedbyagiveninstance.It lookslikethisforthe2.6.14productionkernel: VERSION=2 PATCHLEVEL=6 SUBLEVEL=14 EXTRAVERSION= NAME=AffluentAlbatross Laterinthesamemakefile,thesemacrosareusedtoformaversion-levelmacro,likethis: KERNELRELEASE=$(VERSION).$(PATCHLEVEL).$(SUBLEVEL)$(EXTRAVERSION) Thismacroisusedinseveralplacesinthekernelsourcetreetoindicatethekernelversion.In fact,versioninformationisusedwithsufficientfrequencythatthekerneldevelopershavededicateda set of macros derived from the version macros in the makefile. These macros are found in …/include/linux/version.h[26][2]intheLinuxkernelsourcetree.TheyarereproducedhereasListing 4-1. Listing4-1.KernelincludeFile:…/include/linux/version.h #defineUTS_RELEASE"2.6.14" #defineLINUX_VERSION_CODE132622 #defineKERNEL_VERSION(a,b,c)(((a)<<16)+((b)<<8)+(c)) YoucancheckthekernelversionfromacommandpromptonarunningLinuxsystemlikethis: $cat/proc/version Linuxversion2.6.13(chris@pluto)(gccversion4.0.0(DENXELDK4.04.0.0))#2 ThuFeb1619:30:13EST2006 Onefinalnoteaboutkernelversions:Youcanmakeiteasytokeeptrackofthekernelversionin yourownkernelprojectbycustomizingtheEXtrAVERSIONfield. Forexample,ifyouweredevelopingenhancementsforsomenewkernelfeature,youmightset EXtrAVERSIONtosomethinglikethis: EXTRAVERSION=-foo Later,whenyouusecat/proc/version,youwouldseeLinuxversion2.6.13-foo,andthiswould helpyoudistinguishbetweendevelopmentversionsofyourownkernel. 4.1.2.KernelSourceRepositories Theofficialhomeforthekernelsourcecodeiswww.kernel.org.Thereyoucanfindbothcurrent andhistoricalversionsoftheLinuxkernel,aswellasnumerouspatches.TheprimaryFTPrepository foundatftp.kernel.orgcontainssubdirectoriesgoingallthewaybacktoLinuxVersion1.0.Thissite istheprimaryfocusfortheongoingdevelopmentactivitieswithintheLinuxkernel. IfyoudownloadarecentLinuxkernelfromkernel.org,youwillfindfilesinthesourcetreefor 25 different architectures and subarchitectures. Several other development trees support the major architectures.Oneofthereasonsissimplythesheervolumeofdevelopersandchangestothekernel. Ifeverydeveloperoneveryarchitecturesubmittedpatchestokernel.org,themaintainerswouldbe inundatedwithchangesandpatchmanagement,andwouldnevergettodoanyfeaturedevelopment. Asanyoneinvolvedwithkerneldevelopmentwilltellyou,it'salreadyverybusy! Several other public source trees exist outside the mainline kernel.org source, mostly for architecture-specificdevelopment.Forexample,adeveloperworkingontheMIPSarchitecturemight find a suitable kernel at www.linux-mips.org. Normally, work done in an architecture tree is eventually submittedtothekernel.orgkernel.Mostarchitecturedeveloperstrytosyncuptothemainlinekernel often, to keep up with new developments whenever possible. However, it is not always straightforwardtogetone'spatchesincludedinthemainlinekernel,andtherewillalwaysbealag. Indeed,differencesinthearchitecturekerneltreesexistatanygivenpointintime. Ifyouarewonderinghowtofindakernelforyourparticularapplication,thebestwaytoproceed istoobtainthelateststableLinuxsourcetree.Checktoseeifsupportforyourparticularprocessor exists, and then search the Linux kernel mailing lists for any patches or issues related to your application.Alsofindthemailinglistthatmostcloselymatchesyourinterest,andsearchthatarchive also. Appendix E, "Open Source Resources," contains several good references and sources of informationrelatedtokernelsourcerepositories,mailinglists,andmore. 4.2.LinuxKernelConstruction Inthenextfewsections,weexplorethelayout,organization,andconstructionoftheLinuxkernel. Armedwiththisknowledge,youwillfinditmucheasiertonavigatethislarge,complexsourcecode base. Over time, there have been significant improvements in the organization of the source tree, especiallyinthearchitecturebranch,whichcontainssupportfornumerousarchitecturesandspecific machines. As this book is being written, an effort is underway to collapse the ppc and ppc64 architecture branches into a single common powerpc branch. When the dust settles, there will be many improvements, including elimination of duplicate code, better organization of files, and partitioningoffunctionality. 4.2.1.Top-LevelSourceDirectory Wemakefrequentreferencetothetop-levelsourcedirectorythroughoutthebook.Ineverycase, we are referring to the highest-level directory contained in the kernel source tree. On any given machine, it might be located anywhere, but on a desktop Linux workstation, it is often found in /usr/src/linux-x.y.z, where x.y.z represents the kernel version. Throughout the book, we use the shorthand.../torepresentthetop-levelkernelsourcedirectory. Thetop-levelkernelsourcedirectorycontainsthefollowingsubdirectories.(Wehaveomittedthe nondirectory entries in this listing, as well as directories used for source control for clarity and brevity.) archcryptoDocumentationdriversfsincludeinitipckernellibmmnetscriptssecuritysoundusr Many of these subdirectories contain several additional levels of subdirectories containing sourcecode,makefiles,andconfigurationfiles.ByfarthelargestbranchoftheLinuxkernelsource tree is found under .../drivers. Here you can find support for Ethernet network cards, USB controllers,andthenumeroushardwaredevicesthattheLinuxkernelsupports.Asyoumightimagine, the .../arch subdirectory is the next largest, containing support for more than 20 unique processor architectures. Additionalfilesfoundinthetop-levelLinuxsubdirectoryincludethetop-levelmakefile,ahidden configuration file (dot-config, introduced in Section 4.3.1, "The Dot-Config") and various other informationalfilesnotinvolvedinthebuilditself.Finally,twoimportantbuildtargetsarefoundin thetop-levelkernelsourcetreeafterasuccessfulbuild:System.mapandthekernelproper,vmlinux. Botharedescribedshortly. 4.2.2.CompilingtheKernel Understanding alargebodyofsoftwaresuchasLinuxcanbeadauntingtask.Itistoolargeto simply"stepthrough"thecodetofollowwhatishappening.Multithreadingandpreemptionaddtothe complexityofanalysis.Infact,evenlocatingtheentrypoint(thefirstlineofcodetobeexecutedupon entrytothekernel)canbechallenging.Oneofthemoreusefulwaystounderstandthestructureofa largebinaryimageistoexamineitsbuiltcomponents. The output of the kernel build system produces several common files, as well as one or more architecture-specificbinarymodules.Commonfilesarealwaysbuiltregardlessofthearchitecture. Two of the common files are System.map and vmlinux, introduced earlier. The former is useful during kernel debug and is particularly interesting. It contains a human-readable list of the kernel symbols and their respective addresses. The latter is an architecture-specific ELF[27] file in executableformat.Itisproducedbythetop-levelkernelmakefileforeveryarchitecture.Ifthekernel was compiled with symbolic debug information, it will be contained in the vmlinux image. In practice,althoughitisanELFexecutable,thisfileisusuallyneverbooteddirectly,asyouwillsee shortly. Listing4-2isasnippetofoutputresultingfromexecutingmakeinarecentkerneltreeconfigured for the ARM XScale architecture. The kernel source tree was configured for the ADI Engineering CoyotereferenceboardbasedontheIntelIXP425networkprocessorusingthefollowingcommand: makeARCH=armCROSS_COMPILE=xscale_be-ixp4xx_defconfig This command does not build the kernel; it prepares the kernel source tree for the XScale architectureincludinganinitialdefaultconfigurationforthisarchitectureandprocessor.Itbuildsa defaultconfiguration(thedot-configfile)thatdrivesthekernelbuild,basedonthedefaultsfoundin theixp4xx_defconfigfile.Wehavemoretosayabouttheconfigurationprocesslater,inSection4.3, "KernelBuildSystem." Listing4-2showsthecommandthatbuildsthekernel.Onlythefirstfewandlastfewlinesofthe buildoutputareshownforthisdiscussion. Listing4-2.KernelBuildOutput $makeARCH=armCROSS_COMPILE=xscale_be-zImage CHKinclude/linux/version.h HOSTCCscripts/basic/fixdep . .<hundredsoflinesofoutputomittedhere> . LDvmlinux SYSMAPSystem.map SYSMAP.tmp_System.map OBJCOPYarch/arm/boot/Image Kernel:arch/arm/boot/Imageisready ASarch/arm/boot/compressed/head.o GZIParch/arm/boot/compressed/piggy.gz ASarch/arm/boot/compressed/piggy.o CCarch/arm/boot/compressed/misc.o ASarch/arm/boot/compressed/head-xscale.o ASarch/arm/boot/compressed/big-endian.o LDarch/arm/boot/compressed/vmlinux OBJCOPYarch/arm/boot/zImage Kernel:arch/arm/boot/zImageisready Buildingmodules,stage2. ... Tobegin,noticetheinvocationofthebuild.Boththedesiredarchitecture(ARCH=arm)andthe toolchain(CROSS_COMPILE=xscale_be-)werespecifiedonthecommandline.Thisforcesmaketo usetheXScaletoolchaintobuildthekernelimageandtousethearm-specificbranchofthekernel sourcetreeforarchitecture-dependentportionsofthebuild.WealsospecifyatargetcalledzImage. ThistargetiscommontomanyarchitecturesandisdescribedinChapter5,"KernelInitialization." Thenextthingyoumightnoticeisthattheactualcommandsusedforeachstephavebeenhidden andreplacedwithashorthandnotation.Themotivationbehindthiswastocleanupthebuildoutputto draw more attention to intermediate build issues, particularly compiler warnings. In earlier kernel source trees, each compilation or link command was output to the console verbosely, which often requiredseverallinesforeachstep.Theendresultwasvirtuallyunreadable,andcompilerwarnings slippedbyunnoticedinthenoise.Thenewsystemisdefinitelyanimprovementbecauseanyanomaly in the build process is easily spotted. If you want or need to see the complete build step, you can forceverboseoutputbydefiningV=1onthemakecommandline. We have omitted most of the actual compilation and link steps in Listing 4-2, for clarity. (This particularbuildhasmorethan900individualcompileandlinkcommandsinthebuild.Thatwould havemadeforalonglisting,indeed.)Afteralltheintermediatefilesandlibraryarchiveshavebeen builtandcompiled,theyareputtogetherinonelargeELFbuildtargetcalledvmlinux.Althoughitis architecture specific, this vmlinux target is a common targetit is produced for all supported Linux architectures. 4.2.3.TheKernelProper:vmlinux NoticethislineinListing4-2: LD/arch/arm/boot/compressed/vmlinux The vmlinux file is the actual kernel proper. It is a fully stand-alone, monolithic image. No unresolvedexternalreferencesexistwithinthevmlinuxbinary.Whencausedtoexecuteintheproper context(byabootloaderdesignedtoboottheLinuxkernel),itbootstheboardonwhichitisrunning, leavingacompletelyfunctionalkernel. Inkeepingwiththephilosophythattounderstandasystemonemustfirstunderstanditsparts,let's lookattheconstructionofthevmlinuxkernelobject.Listing4-3reproducestheactuallinkstageof the build process that resulted in the vmlinux ELF object. We have formatted it with line breaks (indicatedbytheUNIXline-continuationcharacter,'\')tomakeitmorereadable,butotherwiseitis theexactoutputproducedbythevmlinuxlinkstepinthebuildprocessfromListing4-2.Ifyouwere buildingthekernelbyhand,thisisthelinkcommandyouwouldissuefromthecommandline. Listing4-3.LinkStage:vmlinux xscale_be-ld-EB-p--no-undefined-X-ovmlinux\ -Tarch/arm/kernel/vmlinux.lds\ arch/arm/kernel/head.o\ arch/arm/kernel/init_task.o\ init/built-in.o\ --start-group\ usr/built-in.o\ arch/arm/kernel/built-in.o\ arch/arm/mm/built-in.o\ arch/arm/common/built-in.o\ arch/arm/mach-ixp4xx/built-in.o\ arch/arm/nwfpe/built-in.o\ kernel/built-in.o\ mm/built-in.o\ fs/built-in.o\ ipc/built-in.o\ security/built-in.o\ crypto/built-in.o\ lib/lib.a\ arch/arm/lib/lib.a\ lib/built-in.o\ arch/arm/lib/built-in.o\ drivers/built-in.o\ sound/built-in.o\ net/built-in.o\ --end-group\ .tmp_kallsyms2.o 4.2.4.KernelImageComponents From Listing 4-3, you can see that the vmlinux image consists of several composite binary images. Right now, it is not important to understand the purpose of each component. What is importantistounderstandthetop-levelviewofwhatcomponentsmakeupthekernel.Thefirstline ofthelinkcommandinListing4-3specifiestheoutputfile(-ovmlinux.)Thesecondlinespecifies thelinkerscriptfile(-Tvmlinux.lds),adetailedrecipeforhowthekernelbinaryimageshouldbe linked.[28] ThethirdandsubsequentlinesfromListing4-3specifytheobjectmodulesthatformtheresulting binary image. Notice that the first object specified is head.o. This object was assembled from /arch/arm/kernel/head.S, an architecture-specific assembly language source file that performs very low-levelkernelinitialization.Ifyouweresearchingforthefirstlineofcodetobeexecutedbythe kernel,itwouldmakesensetostartyoursearchherebecauseitwillultimatelybethefirstcodefound inthebinaryimagecreatedbythislinkstage.WeexaminekernelinitializationindetailinChapter5. The next object, init_task.o, sets up initial thread and task structures that the kernel requires. Followingthisisalargecollectionofobjectmodules,eachhavingacommonname:built-in.o.You willnotice,however,thateachbuilt-in.oobjectcomesfromaspecificpartofthekernelsourcetree, asindicatedbythepathcomponentprecedingthebuilt-in.oobjectname.Thesearethebinaryobjects thatareincludedinthekernelimage.Anillustrationmightmakethisclearer. Figure4-1illustratesthebinarymakeupofthevmlinuximage.Itcontainsasectionforeachline ofthelinkstage.It'snottoscalebecauseofspaceconsiderations,butyoucanseetherelativesizesof eachfunctionalcomponent. Figure4-1.vmlinuximagecomponents Itmightcomeasnosurprisethatthethreelargestbinarycomponentsarethefilesystemcode,the network code, andall thebuilt-indrivers.If youtakethekernel code andthearchitecture-specific kernelcodetogether,thisisthenext-largestbinarycomponent.Hereyoufindthescheduler,process andthreadmanagement,timermanagement,andothercorekernelfunctionality.Naturally,thekernel contains some architecture-specific functionality, such as low-level context switching, hardwarelevel interrupt and timer processing, processor exception handling, and more. This is found in .../arch/arm/kernel. Bear in mind that we are looking at a specific example of a kernel build. In this particular example,wearebuildingakernelspecifictotheARMXScalearchitectureand,morespecifically, the Intel IXP425 network processor on the ADI Engineering reference board. You can see the machine-specific binarycomponentsin Figure4-1 asarch/arm/mach-ixp4xx.Eacharchitectureand machinetype(processor/referenceboard)hasdifferentelementsinthearchitecture-specificportions of the kernel, so the makeup of the vmlinux image is slightly different. When you understand one example,youwillfinditeasytonavigateothers. Tohelpyouunderstandthebreakdownoffunctionalityinthekernelsourcetree,Table4-1lists eachcomponentinFigure4-1,togetherwithashortdescriptionofeachbinaryelementthatmakesup thevmlinuximage. Table4-1.vmlinuxImageComponentsDescription Component Description arch/arm/kernel/head.o Kernelarchitecture-specificstartupcode. init_task.o Initialthreadandtaskstructsrequiredbykernel. init/built-in.o Mainkernel-initializationcode.SeeChapter5. usr/built-in.o Built-ininitramfsimage.SeeChapter5. arch/arm/kernel/built-in.o Architecture-specifickernelcode. arch/arm/mm/built-in.o Architecture-specificmemory-managementcode. arch/arm/common/built-in.o Architecture-specificgenericcode.Variesbyarchitecture. arch/arm/mach-ixp4xx/built-in.o Machine-specificcode,usuallyinitialization. arch/arm/nwfpe/built-in.o Architecture-specificfloatingpoint-emulationcode. kernel/built-in.o Commoncomponentsofthekernelitself. mm/built-in.o Commoncomponentsofmemory-managementcode. ipc/built-in.o Interprocesscommunications,suchasSysVIPC. security/built-in.o Linuxsecuritycomponents. lib/lib.a Archiveofmiscellaneoushelperfunctions. arch/arm/lib/lib.a Architecture-specificcommonfacilities.Variesbyarchitecture. lib/built-in.o Commonkernelhelperfunctions. drivers/built-in.o Allthebuilt-indriversnotloadablemodules. sound/built-in.o Sounddrivers. net/built-in.o Linuxnetworking. .tmp_kallsyms2.o Symboltable. When we speak of the kernel proper, this vmlinux image is being referenced. As mentioned earlier, very few platforms boot this image directly. For one thing, it is almost universally compressed.Atabareminimum,abootloadermustdecompresstheimage.Manyplatformsrequire sometypeofstubboltedontotheimagetoperformthedecompression.LaterinChapter5,youwill learnhowthisimageispackagedfordifferentarchitectures,machinetypes,andbootloaders,andthe requirementsforbootingit. 4.2.5.SubdirectoryLayout Now that you've seen how the build system controls the kernel image, let's take a look at a representativekernelsubdirectory.Listing4-4detailsthecontentsofthemach-ixp425subdirectory. Thisdirectoryexistsunderthe.../arch/armarchitecture-specificbranchofthesourcetree. Listing4-4.KernelSubdirectory $ls-llinux-2.6/arch/arm/mach-ixp425 total92 -rw-rw-r--1chrischris11892Oct1014:53built-in.o -rw-rw-r--1chrischris6924Sep2915:39common.c -rw-rw-r--1chrischris3525Oct1014:53common.o -rw-rw-r--1chrischris13062Sep2915:39common-pci.c -rw-rw-r--1chrischris7504Oct1014:53common-pci.o -rw-rw-r--1chrischris1728Sep2915:39coyote-pci.c -rw-rw-r--1chrischris1572Oct1014:53coyote-pci.o -rw-rw-r--1chrischris2118Sep2915:39coyote-setup.c -rw-rw-r--1chrischris2180Oct1014:53coyote-setup.o -rw-rw-r--1chrischris2042Sep2915:39ixdp425-pci.c -rw-rw-r--1chrischris3656Sep2915:39ixdp425-setup.c -rw-rw-r--1chrischris2761Sep2915:39Kconfig -rw-rw-r--1chrischris259Sep2915:39Makefile -rw-rw-r--1chrischris3102Sep2915:39prpmc1100-pci.c The directory contents in Listing 4-4 have common components found in many kernel source subdirectories: Makefile and Kconfig. These two files drive the kernel configuration-and-build process.Let'slookathowthatworks. 4.3.KernelBuildSystem TheLinuxkernelconfigurationandbuildsystemisrathercomplicated,asonewouldexpectof software projects containing more than six million lines of code! In this section, we cover the foundationofthekernelbuildsystemfordeveloperswhoneedtocustomizethebuildenvironment. ArecentLinuxkernelsnapshotshowedmorethan800makefiles[29]inthekernelsourcetree.This mightsoundlikealargenumber,butitmightnotseemsolargewhenyouunderstandthestructureand operationofthebuildsystem.TheLinuxkernelbuildsystemhasbeensignificantlyupdatedsincethe daysofLinux2.4andearlier.Forthoseofyoufamiliarwiththeolderkernelbuildsystem,we'resure you will find the new Kbuild system to be a huge improvement. We limit our discussion in this sectiontothisandlaterkernelversionsbasedonKbuild. 4.3.1.TheDot-Config Introduced earlier, the dot-config file is the configuration blueprint for building a Linux kernel image. You will likely spend significant effort at the start of your Linux project building a configuration that is appropriate for your embedded platform. Several editors, both text based and graphical,aredesignedtoedityourkernelconfiguration.Theoutputofthisconfigurationexerciseis written to a configuration file named .config, located in the top-level Linux source directory that drivesthekernelbuild. Youhavelikelyinvestedsignificanttimeperfectingyourkernelconfiguration,soyouwillwantto protectit.Severalmakecommandsdeletethisconfigurationfilewithoutwarning.Themostcommon is make mrproper. This make target is designed to return the kernel source tree to its pristine, unconfigured state. This includes removing all configuration data from the source treeand, yes, it deletesyour.config. As you might know, any filename in Linux preceded by a dot is a hidden file in Linux. It is unfortunatethatsuchanimportantfileismarkedhidden;thishasbroughtconsiderablegrieftomore thanonedeveloper.Ifyouexecutemakemrproperwithouthavingabackupcopyofyour.configfile, you,too,willshareourgrief.(Youhavebeenwarnedbackupyour.configfile!) The.configfileisacollectionofdefinitionswithasimpleformat.Listing4.5showsasnippetof a.configfromarecentLinuxkernelrelease. Listing4-5.SnippetfromLinux2.6.config ... #USBsupport # CONFIG_USB=m #CONFIG_USB_DEBUGisnotset #MiscellaneousUSBoptions # CONFIG_USB_DEVICEFS=y #CONFIG_USB_BANDWIDTHisnotset #CONFIG_USB_DYNAMIC_MINORSisnotset #USBHostControllerDrivers # CONFIG_USB_EHCI_HCD=m #CONFIG_USB_EHCI_SPLIT_ISOisnotset #CONFIG_USB_EHCI_ROOT_HUB_TTisnotset CONFIG_USB_OHCI_HCD=m CONFIG_USB_UHCI_HCD=m ... Tounderstandthe.configfile,youneedtounderstandafundamentalaspectoftheLinuxkernel. Linux has a monolithic structure. That is, the entire kernel is compiled and linked as a single staticallylinkedexecutable.However,itispossibletocompileandincrementallylink[30]aset of sourcesintoasingleobjectmodulesuitablefordynamicinsertionintoarunningkernel.Thisisthe usual method for supporting most common device drivers. In Linux, these are called loadable modules. They are also generically called device drivers. After the kernel is booted, a special applicationprogramisinvokedtoinserttheloadablemoduleintoarunningkernel. Armedwiththatknowledge,let'slookagainatListing4-5.Thissnippetoftheconfigurationfile (.config) shows a portion of the USB subsystem configuration. The first configuration option, CONFIG_USB=m,declaresthattheUSBsubsystemistobeincludedinthiskernelconfigurationand thatitwillbecompiled asadynamically loadablemodule (=m), to be loaded sometime after the kernelhasbooted.Theotherchoicewouldhavebeen=y,inwhichcasetheUSBmodulewouldbe compiled and statically linked as part of the kernel image itself. It would end up in the .../drivers/built-in.ocompositebinarythatyousawinListing4-3andFigure4-1.Theastutereader willrealizethatifadriverisconfiguredasaloadablemodule,itscodeisnotincludedinthekernel proper,butratherexistsasastand-aloneobjectmodule,aloadablemodule,tobeinsertedintothe runningkernelafterboot. Notice in Listing 4-5 the CONFIG_USB_DEVICEFS=y declaration. This configuration option behaves in a slightly different manner. In this case, USB_DEVICEFS (as configuration options are commonlyabbreviated)isnotastand-alonemodule,butratherafeaturetobeenabledordisabledin theUSBdriver.Itdoesnotnecessarilyresultinamodulethatiscompiledintothekernelproper(=y); instead, it enables one or more features, often represented as additional object modules to be includedintheoverallUSBdevicedrivermodule.Usually,thehelptextintheconfigurationeditor, orthehierarchypresentedbytheconfigurationeditor,makesthisdistinctionclear. 4.3.2.ConfigurationEditor(s) Early kernels used a simple command line driven script to configure the kernel. This was cumbersome even for early kernels, in which the number of configuration parameters was much smaller.Thiscommandlinestyleinterfaceisstillsupported,butusingitistedious,tosaytheleast.A typical configuration from a recent kernel requires answering more than 600 questions from the command line, entering your choice followed by the Enter key for each query from the script. Furthermore, if you make a mistake, there is no way to back up; you must start from the beginning again.Thatcanbeprofoundlyfrustratingifyoumakeamistakeonthe599thentry! Insomesituations,suchasbuildingakernelonanembeddedsystemwithoutgraphics,usingthe commandlineconfigurationutilityisunavoidable,butthisauthorwouldgotogreatlengthstofinda wayaroundit. The kernel-configuration subsystem uses several graphical front ends. In fact, a recent Linux kernel release included 10 such configuration targets. They are summarized here, from text taken directlyfromtheoutputofmakehelp: •configUpdatecurrentconfigusingaline-orientedprogram •menuconfigUpdatecurrentconfigusingamenu-basedprogram •xconfigUpdatecurrentconfigusingaQT-basedfrontend •gconfigUpdatecurrentconfigusingaGTK-basedfrontend •oldconfigUpdatecurrentconfigusingaprovided.configasthebase •randconfigNewconfigwithrandomanswertoalloptions •defconfigNewconfigwithdefaultanswertoalloptions •allmodconfigNewconfigthatselectsmodules,whenpossible •allyesconfigNewconfiginwhichalloptionsareacceptedwithyes •allnoconfigNewminimalconfig The first four of these makefile configuration targets invoke a form of configuration editor, as describedinthelist.Becauseofspaceconsiderations,wefocusourdiscussioninthischapterand othersonlyontheGTK-basedgraphicalfrontend.Realizethatyoucanusetheconfigurationeditorof yourchoicewiththesameresults. The configuration editor is invoked by entering the command make gconfig from the top-level kernel directory.[31] Figure 4-2 shows the top-level configuration menu presented to the developer whengconfigisrun.Fromhere,everyavailableconfigurationparametercanbeaccessedtogenerate acustomkernelconfiguration. Figure4-2.Top-levelkernelconfiguration Whentheconfigurationeditorisexited,youarepromptedtosaveyourchanges.Ifyouelectto saveyourchanges,theglobalconfigurationfile.configisupdated(orcreated,ifitdoesnotalready exist). This .config file, introduced earlier, drives the kernel build via the top-level makefile. You willnoticeinthismakefilethatthe.configfileisreaddirectlybyanincludestatement. Most kernel software modules also read the configuration indirectly via the .config file as follows. During the build process, the .config file is processed into a C header file found in the .../include/linuxdirectory,calledautoconf.h.Thisisanautomaticallygeneratedfileandshouldnever beediteddirectlybecauseeditsarelosteachtimeaconfigurationeditorisrun.Manykernelsource files include this file directly using the #include preprocessor directive. Listing 4-6 reproduces a section of this header file that corresponds to the earlier USB example above. Note that, for each entryinthe.configfilesnippetinListing4-5,acorrespondingentryiscreatedinautoconf.h.Thisis howthesourcefilesinthekernelsourcetreereferencethekernelconfiguration. Listing4-6.Linuxautoconf.h /* *USBsupport */ #defineCONFIG_USB_MODULE1 #undefCONFIG_USB_DEBUG /* *MiscellaneousUSBoptions */ #defineCONFIG_USB_DEVICEFS1 #undefCONFIG_USB_BANDWIDTH #undefCONFIG_USB_DYNAMIC_MINORS /* *USBHostControllerDrivers */ #defineCONFIG_USB_EHCI_HCD_MODULE1 #undefCONFIG_USB_EHCI_SPLIT_ISO #undefCONFIG_USB_EHCI_ROOT_HUB_TT #defineCONFIG_USB_OHCI_HCD_MODULE1 #defineCONFIG_USB_UHCI_HCD_MODULE1 Ifyou haven'talreadydoneso,executemake gconfiginyourtop-levelkernelsourcedirectory, and poke around this configuration utility to see the large number of subsections and configuration optionsavailabletotheLinuxdeveloper.Aslongasyoudon'texplicitlysaveyourchanges,theyare lostuponexitingtheconfigurationeditorandyoucansafelyexplorewithoutmodifyingyourkernel configuration.[32]Manyconfigurationparameterscontain helpfulexplanationtext, whichcanaddto yourunderstandingofthedifferentconfigurationoptions. 4.3.3.MakefileTargets If you type make help at the top-level Linux source directory, you are presented with a list of targets that can be generated from the source tree. The most common use of make is to specify no target. This generates the kernel ELF file vmlinux and is the default binary image for your chosen architecture (for example, bzImage for x86). Specifying make with no target also builds all the device-drivermodules(kernel-loadablemodules)specifiedbytheconfiguration. Many architectures and machine types require binary targets specific to the architecture and bootloader in use. One of the more common architecture specific targets is zImage. In many architectures, this is the default target image that can be loaded and run on the target embedded system.OneofthecommonmistakesthatnewcomersmakeistospecifybzImageasthemaketarget. ThebzImagetargetisspecifictothex86/PCarchitecture.Contrarytopopularmyth,thebzImageis not a bzip2 -compressed image. It is a big zImage. Without going into the details of legacy PC architecture,itisenoughtoknowthatabzImageissuitableonlyforPC-compatiblemachineswithan industry-standardPC-styleBIOS. Listing4-7containstheoutputfrommakehelpfromarecentLinuxkernel.Youcanseefromthe listingthatmanytargetsareavailableinthetop-levelLinuxkernelmakefile.Eachislistedalongwith ashortdescriptionofitsuse.Itisimportanttorealizethateventhehelpmaketarget(asinmakehelp) is architecture specific. You get a different list of architecture-specific targets depending on the architectureyoupassonthemakeinvocation.Listing4-7illustratesaninvocationthatspecifiesthe ARMarchitecture,asyoucanseefromthemakecommandline. Listing4-7.MakefileTargets $makeARCH=armhelp Cleaningtargets: clean-removemostgeneratedfilesbutkeeptheconfig mrproper-removeallgeneratedfiles+config+variousbackupfiles Configurationtargets: config-Updatecurrentconfigutilisingaline-orientedprogram menuconfig-Updatecurrentconfigutilisingamenubasedprogram xconfig-UpdatecurrentconfigutilisingaQTbasedfront-end gconfig-UpdatecurrentconfigutilisingaGTKbasedfront-end oldconfig-Updatecurrentconfigutilisingaprovided.configasbase randconfig-Newconfigwithrandomanswertoalloptions defconfig-Newconfigwithdefaultanswertoalloptions allmodconfig-Newconfigselectingmoduleswhenpossible allyesconfig-Newconfigwherealloptionsareacceptedwithyes allnoconfig-Newminimalconfig Othergenerictargets: all-Buildalltargetsmarkedwith[*] *vmlinux-Buildthebarekernel *modules-Buildallmodules modules_install-Installallmodules dir/-Buildallfilesindirandbelow dir/file.[ois]-Buildspecifiedtargetonly dir/file.ko-Buildmoduleincludingfinallink rpm-BuildakernelasanRPMpackage tags/TAGS-Generatetagsfileforeditors cscope-Generatecscopeindex kernelrelease-Outputthereleaseversionstring Staticanalysers buildcheck-Listdanglingreferencestovmlinuxdiscardedsectionsand initsectionsfromnon-initsections checkstack-Generatealistofstackhogs namespacecheck-Namespaceanalysisoncompiledkernel Kernelpackaging: rpm-pkg-BuildthekernelasanRPMpackage binrpm-pkg-Buildanrpmpackagecontainingthecompiledkerneland modules deb-pkg-Buildthekernelasandebpackage tar-pkg-Buildthekernelasanuncompressedtarball targz-pkg-Buildthekernelasagzipcompressedtarball tarbz2-pkg-Buildthekernelasabzip2compressedtarball Documentationtargets: Linuxkernelinternaldocumentationindifferentformats: xmldocs(XMLDocBook),psdocs(Postscript),pdfdocs(PDF) htmldocs(HTML),mandocs(manpages,useinstallmandocstoinstall) Architecturespecifictargets(arm): *zImage-Compressedkernelimage(arch/arm/boot/zImage) Image-Uncompressedkernelimage(arch/arm/boot/Image) *xipImage-XIPkernelimage,ifconfigured(arch/arm/boot/xipImage) bootpImage-CombinedzImageandinitialRAMdisk (supplyinitrdimageviamakevariableINITRD=<path>) install-Installuncompressedkernel zinstall-Installcompressedkernel Installusing(your)~/bin/installkernelor (distribution)/sbin/installkernelor installto$(INSTALL_PATH)andrunlilo assabet_defconfig-Buildforassabet badge4_defconfig-Buildforbadge4 bast_defconfig-Buildforbast cerfcube_defconfig-Buildforcerfcube clps7500_defconfig-Buildforclps7500 collie_defconfig-Buildforcollie corgi_defconfig-Buildforcorgi ebsa110_defconfig-Buildforebsa110 edb7211_defconfig-Buildforedb7211 enp2611_defconfig-Buildforenp2611 ep80219_defconfig-Buildforep80219 epxa10db_defconfig-Buildforepxa10db footbridge_defconfig-Buildforfootbridge fortunet_defconfig-Buildforfortunet h3600_defconfig-Buildforh3600 h7201_defconfig-Buildforh7201 h7202_defconfig-Buildforh7202 hackkit_defconfig-Buildforhackkit integrator_defconfig-Buildforintegrator iq31244_defconfig-Buildforiq31244 iq80321_defconfig-Buildforiq80321 iq80331_defconfig-Buildforiq80331 iq80332_defconfig-Buildforiq80332 ixdp2400_defconfig-Buildforixdp2400 ixdp2401_defconfig-Buildforixdp2401 ixdp2800_defconfig-Buildforixdp2800 ixdp2801_defconfig-Buildforixdp2801 ixp4xx_defconfig-Buildforixp4xx jornada720_defconfig-Buildforjornada720 lart_defconfig-Buildforlart lpd7a400_defconfig-Buildforlpd7a400 lpd7a404_defconfig-Buildforlpd7a404 lubbock_defconfig-Buildforlubbock lusl7200_defconfig-Buildforlusl7200 mainstone_defconfig-Buildformainstone mx1ads_defconfig-Buildformx1ads neponset_defconfig-Buildforneponset netwinder_defconfig-Buildfornetwinder omap_h2_1610_defconfig-Buildforomap_h2_1610 pleb_defconfig-Buildforpleb poodle_defconfig-Buildforpoodle pxa255-idp_defconfig-Buildforpxa255-idp rpc_defconfig-Buildforrpc s3c2410_defconfig-Buildfors3c2410 shannon_defconfig-Buildforshannon shark_defconfig-Buildforshark simpad_defconfig-Buildforsimpad smdk2410_defconfig-Buildforsmdk2410 spitz_defconfig-Buildforspitz versatile_defconfig-Buildforversatile makeV=0|1[targets]0=>quietbuild(default),1=>verbosebuild makeO=dir[targets]Locatealloutputfilesin"dir",including.config makeC=1[targets]Checkallcsourcewith$CHECK(sparse) makeC=2[targets]Forcecheckofallcsourcewith$CHECK(sparse) Execute"make"or"makeall"tobuildalltargetsmarkedwith[*] Forfurtherinfoseethe./READMEfile Manyofthesetargetsyoumightneveruse.However,itisusefultoknowthattheyexist.Asyou canseefromListing4-7,thetargetslistedwithanasteriskarebuiltbydefault.Noticethenumerous defaultconfigurations,listedas*_defconfig.RecallfromSection4.2.2,"CompilingtheKernel,"the command we used to preconfigure a pristine kernel source tree: We invoked make with an architecture and a default configuration. The default configuration was ixp4xx_defconfig, which appears in this list of ARM targets. This is a good way to discover all the default configurations availableforaparticularkernelreleaseandarchitecture. 4.3.4.KernelConfiguration Kconfig(orafilewithasimilarrootfollowedbyanextension,suchasKconfig.ext) existsin almost300kernelsubdirectories.Kconfigdrivestheconfigurationprocessforthefeaturescontained within its subdirectory. The contents of Kconfig are parsed by the configuration subsystem, which presents configuration choices to the user, and contains help text associated with a given configurationparameter. Theconfigurationutility(suchasgconf,presentedearlier)readstheKconfigfilesstartingfrom thearchsubdirectory'sKconfigfile.ItisinvokedfromtheKconfigmakefilewithanentrythatlooks likethis: gconfig:$(obj)/gconf $<arch/$(ARCH)/Kconfig Dependingonwhicharchitectureyouarebuilding,gconfreadsthisarchitecture-specificKconfig as the top-level configuration definition. Contained within Kconfig are a number of lines that look likethis: source"drivers/pci/Kconfig" This directive tells the configuration editor utility to read in another Kconfig file from another location within the kernel source tree. Each architecture contains many such Kconfig files; taken together,thesedeterminethecompletesetofmenuoptionspresentedtotheuserwhenconfiguringthe kernel. Each Kconfig file is free to source additional Kconfig files in different parts of the source tree.Theconfigurationutilitygconf,inthiscase,recursivelyreadstheKconfigfilechainandbuilds theconfigurationmenustructure. Listing4-8isapartialtreeviewoftheKconfigfilesassociatedwiththeARMarchitecture.Ina recentLinux2.6sourcetreefromwhichthisexamplewastaken,thekernelconfigurationwasdefined by170separateKconfigfiles.Thislistingomitsmostofthose,forthesakeofspaceandclaritythe ideaistoshowtheoverallstructure.Tolistthemallinthistreeviewwouldtakeseveralpagesof thistext. Listing4-8.PartialListingofKconfigforARMArchitecture arch/arm/Kconfig<<<<<<(toplevelKconfig) |->init/Kconfig |... |->arch/arm/mach-iop3xx/Kconfig |->arch/arm/mach-ixp4xx/Kconfig |... |->net/Kconfig ||->net/ipv4/Kconfig |||->net/ipv4/ipvs/Kconfig |... |->drivers/char/Kconfig ||->drivers/serial/Kconfig |... |->drivers/usb/Kconfig ||->drivers/usb/core/Kconfig ||->drivers/usb/host/Kconfig |... |->lib/Kconfig LookingatListing4-8,thefilearch/arm/Kconfigwouldcontainalinelikethis: source"net/Kconfig" Thefilenet/Kconfigwouldcontainalinelikethis: source"net/ipv4/Kconfig" …andsoon. As mentioned earlier, these Kconfig files taken together determine the configuration menu structureandconfigurationoptionspresentedtotheuserduringkernelconfiguration.Figure4-3isan exampleoftheconfigurationutility(gconf)fortheARMarchitecturecompiledfromtheexamplein Listing4-8. Figure4-3.gconfconfigurationscreen 4.3.5.CustomConfigurationOptions Many embedded developers add feature support to the Linux kernel to support their particular customhardware.Oneofthemostcommonexamplesofthisismultipleversionsofagivenhardware platform, each of which requires some compile-time options to be configured in the kernel source tree. Instead of having a separate version of the kernel source tree for each hardware version, a developercanaddconfigurationoptionstoenablehiscustomfeatures. Theconfigurationmanagementarchitecturedescribedinthepreviousparagraphsmakesiteasyto customize and add features. A quick peek into a typical Kconfig file shows the structure of the configurationscriptlanguage.Asanexample,assumethatyouhavetwohardwareplatformsbasedon the IXP425 network processor, and that your engineering team had dubbed them Vega and Constellation.Eachboardhasspecializedhardwarethatmustbeinitializedearlyduringthekernel bootphase.Let'sseehoweasyitistoaddtheseconfigurationoptionstothesetofchoicespresented to the developer during kernel configuration. Listing 4-9 is a snippet from the top-level ARM Kconfigfile. Listing4-9.Snippetfrom…/arch/arm/Kconfig source"init/Kconfig" menu"SystemType" choice prompt"ARMsystemtype" defaultARCH_RPC configARCH_CLPS7500 bool"Cirrus-CL-PS7500FE" configARCH_CLPS711X bool"CLPS711x/EP721x-based" ... source"arch/arm/mach-ixp4xx/Kconfig InthisKconfigsnippettakenfromthetop-levelARMarchitectureKconfig,youseethemenuitem SystemTypebeingdefined.AftertheARMSystemtypeprompt,youseealistofchoicesrelatedto the ARM architecture. Later in the file, you see the inclusion of the IXP4 xx -specific Kconfig definitions. In this file, you add your custom configuration switches. Listing 4-10 reproduces a snippet of this file. Again, for readability and convenience, we've omitted irrelevant text, as indicatedbytheellipsis. Listing4-10.FileSnippet:arch/arm/mach-ixp4xx/Kconfig menu"IntelIXP4xxImplementationOptions" comment"IXP4xxPlatforms" configARCH_AVILA bool"Avila" help Say'Y'hereifyouwantyourkerneltosupport... configARCH_ADI_COYOTE bool"Coyote" help Say'Y'hereifyouwantyourkerneltosupport theADIEngineeringCoyote... #(Theseareournewcustomoptions) configARCH_VEGA bool"Vega" help Selectthisoptionfor"Vega"hardwaresupport configARCH_CONSTELLATION bool"Constellation" help Selectthisoptionfor"Constellation" hardwaresupport ... Figure4-4illustratestheresultofthesechangesasitappearswhenrunningthegconfutility(via make ARCH=arm gconfig). As a result of these simple changes, the configuration editor now includes options for our two new hardware platforms.[33] Shortly, you'll see how you can use this configuration information in the source tree to conditionally select objects that contain support for yournewboards. Figure4-4.Customconfigurationoptions Aftertheconfigurationeditor(gconf,intheseexamples)isrunandyouselectsupportforoneof your custom hardware platforms, the .config file introduced earlier contains macros for your new options.Aswithallkernel-configurationoptions,eachisprecededwithCONFIG_toidentifyitasa kernel-configurationoption.Asaresult,twonewconfigurationoptionshavebeendefined,andtheir state has been recorded in the .config file. Listing 4-11 shows the new .config file with your new configurationoptions. Listing4-11.Customized.configFileSnippet ... # #IXP4xxPlatforms # #CONFIG_ARCH_AVILAisnotset #CONFIG_ARCH_ADI_COYOTEisnotset CONFIG_ARCH_VEGA=y #CONFIG_ARCH_CONSTELLATIONisnotset #CONFIG_ARCH_IXDP425isnotset #CONFIG_ARCH_PRPMC1100isnotset ... NoticetwonewconfigurationoptionsrelatedtoyourVegaandConstellationhardwareplatforms. AsillustratedinFigure4-4,youselectedsupportforVega;inthe.configfile,youcanseethenew CONFIG_optionrepresentingthattheVegaboardisselectedandsettothevalue'y'.Noticealso thattheCONFIG_optionrelatedtoConstellationispresentbutnotselected. 4.3.6.KernelMakefiles Whenbuildingthekernel,theMakefilesscantheconfigurationanddecidewhatsubdirectoriesto descendintoandwhatsourcefilestocompileforagivenconfiguration.Tocompletetheexampleof addingsupportfortwocustomhardwareplatforms,VegaandConstellation,let'slookatthemakefile thatwouldreadthisconfigurationandtakesomeactionbasedoncustomizations. Because you're dealing with hardware specific options in this example, assume that the customizations are represented by two hardware-setup modules called vega_setup.c and constellation_setup.c.We'veplacedtheseCsourcefilesinthe.../arch/arm/mach-ixp4xxsubdirectory ofthekernelsourcetree.Listing4-12containsthecompletemakefileforthisdirectoryfromarecent Linuxrelease. Listing4-12.Makefilefrom…/arch/arm/mach-ixp4xxKernelSubdirectory # #Makefileforthelinuxkernel. # obj-y+=common.ocommon-pci.o obj-$(CONFIG_ARCH_IXDP4XX)+=ixdp425-pci.oixdp425-setup.o obj-$(CONFIG_MACH_IXDPG425)+=ixdpg425-pci.ocoyote-setup.o obj-$(CONFIG_ARCH_ADI_COYOTE)+=coyote-pci.ocoyote-setup.o obj-$(CONFIG_MACH_GTWX5715)+=gtwx5715-pci.ogtwx5715-setup.o You might be surprised by the simplicity of this makefile. Much work has gone into the developmentofthekernelbuildsystemforjustthisreason.Fortheaveragedeveloperwhosimply needs to add support for his custom hardware, the design of the kernel build system makes these kindsofcustomizationsverystraightforward.[34] Lookingatthismakefile,itmightbeobviouswhatmustbedonetointroducenewhardwaresetup routinesconditionallybasedonyourconfigurationoptions.Simplyaddthefollowingtwolinesatthe bottomofthemakefile,andyou'redone: obj-$(CONFIG_ARCH_VEGA)+=vega_setup.o obj-$(CONFIG_ARCH_CONSTELLATION)+=costellation_setup.o Thesestepscompletethesimpleadditionofsetupmodulesspecifictothehypotheticalexample customhardware.Usingsimilarlogic,youshouldnowbeabletomakeyourownmodificationstothe kernelconfiguration/buildsystem. 4.3.7.KernelDocumentation AwealthofinformationisavailableintheLinuxsourcetreeitself.Itwouldbedifficultindeedto read it all because there are nearly 650 documentation files in 42 subdirectories in the .../Documentation directory. Be cautious in reading this material: Given the rapid pace of kernel developmentandrelease,thisdocumentationtendstobecomeoutdatedquickly.Nonetheless,itoften providesagreatstartingpointfromwhichyoucanformafoundationonaparticularkernelsubsystem orconcept. Do not neglect the Linux Documentation Project, found at www.tldp.org, where you might find the mostup-to-dateversionofaparticulardocumentormanpage.[35]Thelistofsuggestedreadingatthe endofthischapterduplicatestheURLfortheLinuxDocumentationProject,foreasyreference.Of particular interest to the previous discussion is the Kbuild documentation found in the kernel .../Documentation/kbuildsubdirectory. NodiscussionofKerneldocumentationwouldbecompletewithoutmentioningGoogle.Oneday soon, Googling will appear in Merriam Webster's as a verb! Chances are, many problems and questionsyoumightaskhavealreadybeenaskedandansweredbefore.Spendsometimetobecome proficient in searching the Internet for answers to questions. You will discover numerous mailing listsandotherinformationrepositoriesfullofusefulinformationrelatedtoyourspecificprojector problem.AppendixEcontainsausefullistofopen-sourceresources. 4.4.ObtainingaLinuxKernel Ingeneral,youcanobtainanembeddedLinuxkernelforyourhardwareplatforminthreeways: You can purchase a suitable commercial embedded Linux distribution; you can download a free embeddeddistribution,ifyoucanfindonesuitableforyourparticulararchitectureandprocessor;or youcanfindtheclosestopen-sourceLinuxkerneltoyourapplicationandportityourself.Wediscuss LinuxportinginChapter16,"PortingLinux." Although porting an open source kernel to your custom board is not necessarily difficult, it represents a significant investment in engineering/development resources. This approach gives you access to free software, but deploying Linux in your development project is far from free, as we discussed in Chapter 1, "Introduction." Even for a small system with minimal application requirements,youneedmanymorecomponentsthanjustaLinuxkernel. 4.4.1.WhatElseDoINeed? ThischapterhasfocusedonthelayoutandconstructionoftheLinuxkernelitself.Asyoumight havealreadydiscovered,LinuxisonlyasmallcomponentofanembeddedsystembasedonLinux.In addition to the Linux kernel, you need the following components to develop, test, and launch your embeddedLinuxwidget: •Bootloaderportedtoandconfiguredforyourspecifichardwareplatform •Cross-compilerandassociatedtoolchainforyourchosenarchitecture •Filesystemcontainingmanypackagesbinaryexecutablesandlibrariescompiledforyournative hardwarearchitecture/processor •Devicedriversforanycustomdevicesonyourboard •Developmentenvironment,includinghosttoolsandutilities •Linuxkernelsourcetreeenabledforyourparticularprocessorandboard ThesearethecomponentsofanembeddedLinuxdistribution. 4.5.ChapterSummary • The Linux kernel is more than 10 years old and has become a mainstream, well-supported operatingsystemformanyarchitectures. •TheLinuxopensourcehomeisfoundatwww.kernel.org.Virtuallyeveryreleaseversionofthe kernelisavailablethere,goingallthewaybacktoLinux1.0. •WeleaveittoothergreatbookstodescribethetheoryandoperationoftheLinuxkernel.Here wediscussedhowitisbuiltandidentifiedthecomponentsthatmakeuptheimage.Breakingupthe kernelintounderstandablepiecesisthekeytolearninghowtonavigatethislargesoftwareproject. •Thischaptercoveredthekernelbuildsystemandtheprocessofmodifyingthebuildsystemto facilitatemodifications. • Several kernel configuration editors exist. We chose one and examined how it is driven and howtomodifythemenusandmenuitemswithin.Theseconceptsapplytoallthegraphicalfrontends. • The kernel itself comeswithanentiredirectorystructurefullofusefulkerneldocumentation. Thisisahelpfulresourceforunderstandingandnavigatingthekernelanditsoperation. • This chapter concluded with a brief introduction to the options available for obtaining an embeddedLinuxdistribution. 4.5.1.SuggestionsforAdditionalReading LinuxKernelHOWTO: www.tldp.org/HOWTO/Kernel-HOWTO KernelKbuilddocumentation: http://sourceforge.net/projects/kbuild/ TheLinuxDocumentationProject: www.tldp.org/ ToolInterfaceStandard(TIS)ExecutableandLinkingFormat(ELF)Specification, Version1.2 TISCommittee,May1995 Linuxkernelsourcetree: …/Documentation/kbuild/makefiles.txt Linuxkernelsourcetree: …/Documentation/kbuild/kconfig-language.txt LinuxKernelDevelopment, 2ndEditionRovertLove NovellPress,2005 Chapter5.KernelInitialization When the power is applied to an embedded Linux system, a complex sequence of events is started.Afterafewseconds,theLinuxkernelisoperationalandhasspawnedaseriesofapplication programs as specified by the system init scripts. A significant portion of these activities are governedbysystemconfigurationandareunderthecontroloftheembeddeddeveloper. ThischapterexaminestheinitialsequenceofeventsintheLinuxkernel.Wetakeadetailedlook at the mechanisms and processes used during kernel initialization. We describe the Linux kernel commandlineanditsusetocustomizetheLinuxenvironmentonstartup.Withthisknowledge,you will be able to customize and control the initialization sequence to meet the requirements of your particularembeddedsystem. 5.1.CompositeKernelImage:PiggyandFriends At power-on, the bootloader in an embedded system is first to get processor control. After the bootloader has performed some low-level hardware initialization, control is passed to the Linux kernel.Thiscanbeamanualsequenceofeventstofacilitatethedevelopmentprocess(forexample, the user types interactive load/boot commands at the bootloader prompt), or an automated startup sequencetypicalofaproductionenvironment.WehavededicatedChapter7,"Bootloaders,"tothis subject,sowedeferanydetailedbootloaderdiscussiontothatchapter. In Chapter 4, "The Linux Kernel: A Different Perspective," we examined the components that makeuptheLinuxkernelimage.Recallthatoneofthecommonfilesbuiltforeveryarchitectureisthe ELF binary named vmlinux. This binary file is the monolithic kernel itself, or what we have been callingthekernelproper.Infact,whenwelookedatitsconstructioninthelinkstageofvmlinux,we pointed out where we might look to see where the first line of code might be found. In most architectures, it is found in an assembly language source file called head.S or similar. In the PowerPC (ppc) branch of the kernel, several versions of head.S are present, depending on the processor. For example, the AMCC 440 series processors are initialized from a file called head_44x.S. Somearchitecturesandbootloadersarecapableofdirectlybootingthevmlinuxkernelimage.For example,platformsbasedonPowerPCarchitectureandtheU-Bootbootloadercanusuallybootthe vmlinuximage directly[36] (after conversion from ELF to binary, as you will shortly see). In other combinations of architecture and bootloader, additional functionality might be needed to set up the propercontextandprovidethenecessaryutilitiesforloadingandbootingthekernel. Listing5-1detailsthefinalsequenceofstepsinthekernelbuildprocessforahardwareplatform basedontheADIEngineeringCoyoteReferencePlatform,whichcontainsanIntelIXP425network processor.Thislistingusesthequietformofoutputfromthekernelbuildsystem,whichisthedefault. As pointed out in Chapter 4, it is a useful shorthand notation, allowing more focus on errors and warningsduringthebuild. Listing5-1.FinalKernelBuildSequence:ARM/IXP425(Coyote) $makeARCH=armCROSS_COMPILE=xscale_be-zImage ...<manybuildstepsomittedforclarity> LDvmlinux SYSMAPSystem.map OBJCOPYarch/arm/boot/Image Kernel:arch/arm/boot/Imageisready ASarch/arm/boot/compressed/head.o GZIParch/arm/boot/compressed/piggy.gz ASarch/arm/boot/compressed/piggy.o CCarch/arm/boot/compressed/misc.o ASarch/arm/boot/compressed/head-xscale.o ASarch/arm/boot/compressed/big-endian.o LDarch/arm/boot/compressed/vmlinux OBJCOPYarch/arm/boot/zImage Kernel:arch/arm/boot/zImageisready Buildingmodules,stage2. ... InthethirdlineofListing5-1,thevmlinuximage(thekernelproper)islinked.Followingthat,a number of additional object modules are processed. These include head.o, piggy.o,[37] and the architecture-specifichead-xscale.o,amongothers.(Thetagsidentifywhatishappeningoneachline. Forexample,ASindicatesthattheassemblerisinvoked,GZIPindicatescompression,andsoon.)In general,theseobjectmodulesarespecifictoagivenarchitecture(ARM/XScale,inthisexample)and containlow-levelutilityroutinesneededtobootthekernelonthisparticulararchitecture.Table5-1 detailsthecomponentsfromListing5-1. Table5-1.ARM/XScaleLow-LevelArchitectureObjects Component Function/Description Kernelproper,inELFformat,includingsymbols,comments,debuginfo(ifcompiled vmlinux with-g)andarchitecture-genericcomponents. System.map Text-basedkernelsymboltableforvmlinuxmodule. Image Binarykernelmodule,strippedofsymbols,notes,andcomments. ARM-specificstartupcodegenerictoARMprocessors.Itisthisobjectthatispassed head.o controlbythebootloader. piggy.gz ThefileImagecompressedwithgzip. Thefilepiggy.gzinassemblylanguageformatsoitcanbelinkedwithasubsequent piggy.o object,misc.o(seethetext). Routinesusedfordecompressingthekernelimage(piggy.gz),andthesourceofthe misc.o familiarbootmessage:"UncompressingLinux…Done"onsomearchitectures. headProcessorinitializationspecifictotheXScaleprocessorfamily. xscale.o bigTinyassemblylanguageroutinetoswitchtheXScaleprocessorintobig-endianmode. endian.o Compositekernelimage.Notethisisanunfortunatechoiceofnames,becauseit duplicatesthenameforthekernelproper;thetwoarenotthesame.Thisbinaryimageis vmlinux theresultwhenthekernelproperislinkedwiththeobjectsinthistable.Seethetextfor anexplanation. zImage Finalcompositekernelimageloadedbybootloader.Seethefollowingtext. Anillustrationwillhelp youunderstandthisstructureand thefollowingdiscussion.Figure5-1 shows the image components and their metamorphosis during the build process leading up to a bootablekernelimage.Thefollowingsectionsdescribethecomponentsandprocessindetail. Figure5-1.Compositekernelimageconstruction 5.1.1.TheImageObject AfterthevmlinuxkernelELFfilehasbeenbuilt,thekernelbuildsystemcontinuestoprocessthe targets described in Table 5-1. The Image object is created from the vmlinux object. Image is basicallythevmlinuxELFfilestrippedofredundantsections(notesandcomments)andalsostripped ofanydebuggingsymbolsthatmighthavebeenpresent.Thefollowingcommandisusedforthis: xscale_be-objcopy-Obinary-R.note-R.comment-S\ vmlinuxarch/arm/boot/Image Inthe previousobjcopycommand, the-Ooptiontellsobjcopytogenerateabinaryfile, the-R option removes the ELF sections named .note and .comment, and the -S option is the flag to strip debugging symbols. Notice that objcopy takes the vmlinux ELF image as input and generates the targetbinaryfilecalledImage.Insummary,Imageisnothingmorethanthekernelproperinbinary formstrippedofdebugsymbolsandthe.noteand.commentELFsections. 5.1.2.ArchitectureObjects Following the build sequence further, a number of small modules are compiled. These include several assembly language files (head.o, head-xscale.o, and so on) that perform low-level architecture and processor-specific tasks. Each of these objects is summarized in Table 5-1. Of particularnoteisthesequencecreatingtheobjectcalledpiggy.o.First,theImagefile(binarykernel image)iscompressedusingthisgzipcommand: gzip-f-9<Image>piggy.gz This creates a new file called piggy.gz, which is simply a compressed version of the binary kernelImage.YoucanseethisgraphicallyinFigure5-1.Whatfollowsnextisratherinteresting.An assembly language file called piggy.S is assembled, which contains a reference to the compressed piggy.gz. In essence, the binary kernel image is being piggybacked into a low-level assembly languagebootstraploader.[38]Thisbootstraploaderinitializestheprocessorandrequiredmemory regions,decompressesthebinarykernelimage,andloadsitintotheproperplaceinsystemmemory before passing control to it. Listing 5-2 reproduces .../arch/arm/boot/compressed/piggy.S in its entirety. Listing5-2.AssemblyFilePiggy.S .section.piggydata,#alloc .globlinput_data input_data: .incbin"arch/arm/boot/compressed/piggy.gz" .globlinput_data_end input_data_end:Thissmallassemblylanguagefileissimpleyetproducesacomplexitythatisnot immediatelyobvious.Thepurposeofthisfileistocausethecompressed,binarykernelimagetobe emittedbytheassemblerasanELFsectioncalled.piggydata.Itistriggeredbythe.incbinassembler preprocessor directive, which can be viewed as the assembler's version of a #include file. In summary, the net result of this assembly language file is to contain the compressed binary kernel image as a payload within another imagethe bootstrap loader. Notice the labels input_data and input_data_end.Thebootstraploaderusesthesetoidentifytheboundariesofthebinarypayload,the kernelimage. 5.1.3.BootstrapLoader Not to be confused with a bootloader, many architectures use a bootstrap loader (or secondstageloader)toloadtheLinuxkernelimageintomemory.Somebootstraploadersperformchecksum verificationofthekernelimage,andmostperformdecompressionandrelocationofthekernelimage. Thedifferencebetweenabootloaderandabootstraploaderinthiscontextissimple:Thebootloader controlstheboarduponpower-upanddoesnotrelyontheLinuxkernelinanyway.Incontrast,the bootstraploader'sprimarypurposeinlifeistoactasthegluebetweenaboard-levelbootloaderand theLinuxkernel.Itisthebootstraploader'sresponsibilitytoprovideapropercontextforthekernel torunin,aswellasperformthenecessarystepstodecompressandrelocatethekernelbinaryimage. ItissimilartotheconceptofaprimaryandsecondaryloaderfoundinthePCarchitecture. Figure5-2makesthisconceptclear.Thebootstraploaderisconcatenatedtothekernelimagefor loading. Figure5-2.CompositekernelimageforARMXScale Intheexamplewehavebeenstudying,thebootstraploaderconsistsofthebinaryimagesshown inFigure5-2.Thefunctionsperformedbythisbootstraploaderincludethefollowing: • Low-level assembly processor initialization, which includes support for enabling the processor's internal instruction and data caches, disabling interrupts, and setting up a C runtime environment.Theseincludehead.oandhead-xscale.o. •Decompressionandrelocationcode,embodiedinmisc.o. • Other processor-specific initialization, such as big-endian.o, which enables the big endian modeforthisparticularprocessor. Itisworthnotingthatthedetailswehavebeenexaminingintheprecedingsectionsarespecificto the ARM/XScale kernel implementation. Each architecture has different details, although the conceptsaresimilar.Usingasimilaranalysistothatpresentedhere,youcanlearntherequirements ofyourownarchitecture. 5.1.4.BootMessages Perhapsyou'veseenaPCworkstationbootingadesktopLinuxdistributionsuchasRedHator SUSE Linux. After the PC's own BIOS messages, you see a flurry of console messages being displayedbyLinuxasitinitializesthevariouskernelsubsystems.Significantportionsoftheoutput are common across disparate architectures and machines. Two of the more interesting early boot messages are the kernel version string and the kernel command line, which is detailed shortly. Listing5-3reproducesthekernelbootmessagesfortheADIEngineeringCoyoteReferencePlatform booting Linux on the Intel XScale IXP425 processor. The listing has been formatted with line numbersforeasyreference. Listing5-3.LinuxBootMessagesonIPX425 1UncompressingLinux...done,bootingthekernel. 2Linuxversion2.6.14-clh(chris@pluto)(gccversion3.4.3(MontaVista3.4.3-25.0.30.0501131 2005-07-23))#11SatMar2511:16:33EST2006 3CPU:XScale-IXP42xFamily[690541c1]revision1(ARMv5TE) 4Machine:ADIEngineeringCoyote 5Memorypolicy:ECCdisabled,Datacachewriteback 6CPU0:DVIVTundefined5cache 7CPU0:Icache:32768bytes,associativity32,32bytelines,32sets 8CPU0:Dcache:32768bytes,associativity32,32bytelines,32sets 9Built1zonelists 10Kernelcommandline:console=ttyS0,115200ip=bootproot=/dev/nfs 11PIDhashtableentries:512(order:9,8192bytes) 12Console:colourdummydevice80x30 13Dentrycachehashtableentries:16384(order:4,65536bytes) 14Inode-cachehashtableentries:8192(order:3,32768bytes) 15Memory:64MB=64MBtotal 16Memory:62592KBavailable(1727Kcode,339Kdata,112Kinit) 17Mount-cachehashtableentries:512 18CPU:Testingwritebuffercoherency:ok 19softlockupthread0startedup. 20NET:Registeredprotocolfamily16 21PCI:IXP4xxishost 22PCI:IXP4xxUsingdirectaccessformemoryspace 23PCI:bus0:Fastbacktobacktransfersenabled 24dmabounce:registereddevice0000:00:0f.0onpcibus 25NetWinderFloatingPointEmulatorV0.97(doubleprecision) 26JFFS2version2.2.(NAND)(C)2001-2003RedHat,Inc. 27Serial:8250/16550driver$Revision:1.90$2ports,IRQsharingdisabled 28ttyS0atMMIO0xc8001000(irq=13)isaXScale 29ioschedulernoopregistered 30ioscheduleranticipatoryregistered 31ioschedulerdeadlineregistered 32ioschedulercfqregistered 33RAMDISKdriverinitialized:16RAMdisksof8192Ksize1024blocksize 34loop:loaded(max8devices) 35eepro100.c:v1.09j-t9/29/99DonaldBeckerhttp://www.scyld.com/network/eepro100.html 36eepro100.c:$Revision:1.36$2000/11/17ModifiedbyAndreyV.Savochkin<[email protected] .com.sg>andothers 37eth0:0000:00:0f.0,00:0E:0C:00:82:F8,IRQ28. 38Boardassembly741462-016,Physicalconnectorspresent:RJ45 39Primaryinterfacechipi82555PHY#1. 40Generalself-test:passed. 41Serialsub-systemself-test:passed. 42Internalregistersself-test:passed. 43ROMchecksumself-test:passed(0x8b51f404). 44IXP4XX-Flash.0:Found1x16devicesat0x0in16-bitbank 45Intel/SharpExtendedQueryTableat0x0031 46Usingbufferwritemethod 47cfi_cmdset_0001:Erasesuspendonwriteenabled 48SearchingforRedBootpartitiontableinIXP4XX-Flash.0atoffset0xfe0000 495RedBootpartitionsfoundonMTDdeviceIXP4XX-Flash.0 50Creating5MTDpartitionson"IXP4XX-Flash.0": 510x00000000-0x00060000:"RedBoot" 520x00100000-0x00260000:"MyKernel" 530x00300000-0x00900000:"RootFS" 540x00fc0000-0x00fc1000:"RedBootconfig" 55mtd:partition"RedBootconfig"doesn'tendonaneraseblock--forceread-only0x00fe00000x01000000:"FISdirectory" 56NET:Registeredprotocolfamily2 57IProutecachehashtableentries:1024(order:0,4096bytes) 58TCPestablishedhashtableentries:4096(order:2,16384bytes) 59TCPbindhashtableentries:4096(order:2,16384bytes) 60TCP:Hashtablesconfigured(established4096bind4096) 61TCPrenoregistered 62TCPbicregistered 63NET:Registeredprotocolfamily1 64SendingBOOTPrequests.OK 65IP-Config:GotBOOTPanswerfrom192.168.1.10,myaddressis192.168.1.141 66IP-Config:Complete: 67device=eth0,addr=192.168.1.141,mask=255.255.255.0,gw=255.255.255.255, 68host=192.168.1.141,domain=,nis-domain=(none), 69 bootserver=192.168.1.10, rootserver=192.168.1.10, rootpath=/home/chris/sandbox/coyote-target 70LookingupportofRPC100003/2on192.168.1.10 71LookingupportofRPC100005/1on192.168.1.10 72VFS:Mountedroot(nfsfilesystem). 73Freeinginitmemory:112K 74Mountingproc 75Startingsystemloggers 76Configuringlo 77Startinginetd 78/# Thekernelproducesmuchusefulinformationduringstartup,asshowninListing5-3.Westudy this output in some detail in the next few sections. Line 1 is produced by the bootstrap loader we presentedearlierinthischapter.Thismessagewasproducedbythedecompressionloaderfoundin …/arch/arm/boot/compressed/misc.c. Line2ofListing5-3isthekernelversionstring.Itisthefirstlineofoutputfromthekernelitself. OneofthefirstlinesofCcodeexecutedbythekernel(in.../init/main.c)uponenteringstart_kernel() isasfollows: printk(linux_banner); Thislineproducestheoutputjustdescribedthekernelversionstring,Line2ofListing5-3.This versionstringcontainsanumberofpertinentdatapointsrelatedtothekernelimage: •Kernelversion:Linuxversion2.6.10-clh •Username/machinenamewherekernelwascompiled •Toolchaininfo:gccversion3.4.3,suppliedbyMontaVistaSoftware •Buildnumber •Dateandtimecompiled This is useful information both during development and later in production. All but one of the entries are self-explanatory. The build number is simply a tool that the developers added to the version string to indicate that something more substantial than the date and time changed from one build to the next. It is a way for developers to keep track of the build in a generic and automatic fashion.Youwillnoticeinthisexamplethatthiswastheeleventhbuildinthisseries,asindicatedby the#11online2ofListing5-3.Theversionstringisstoredinahiddenfileinthetop-levelLinux directory and is called .version. It is automatically incremented by a build script found in .../scripts/mkversionandbythetop-levelmakefile.Inshort,itisaversionstringthatisautomatically incrementedwheneveranythingsubstantialinthekernelisrebuilt. 5.2.InitializationFlowofControl Now that we have an understanding of the structure and components of the composite kernel image,let'sexaminetheflowofcontrolfromthebootloadertothekernelinacompletebootcycle. AswediscussedinChapter2,"YourFirstEmbeddedExperience,"thebootloaderisthelow-level component resident in system nonvolatile memory (Flash or ROM) that takes control immediately afterthepowerhasbeenapplied.Itistypicallyasmall,simplesetofroutinesdesignedprimarilyto do low-level initialization, boot image loading, and system diagnostics. It might contain memory dumpandfillroutinesforexaminingandmodifyingthecontentsofmemory.Itmightalsocontainlowlevelboardself-testroutines,includingmemoryandI/Otests.Finally,abootloadercontainslogicfor loadingandpassingcontroltoanotherprogram,usuallyanoperatingsystemsuchasLinux. TheARMXScaleplatformusedasabasisfortheexamplesinthischaptercontainstheRedboot bootloader. When power is first applied, this bootloader is invoked and proceeds to load the operating system (OS). When the bootloader locates and loads the OS image (which could be resident locally in Flash, on a hard drive, or via a local area network or other device), control is passedtothatimage. On this particular XScale platform, the bootloader passes control to our head.o module at the labelStartinthebootstraploader.ThisisillustratedinFigure5-3. Figure5-3.ARMbootcontrolflow As detailed earlier, the bootstrap loader prepended to the kernel image has a single primary responsibility: to create the proper environment to decompress and relocate the kernel, and pass controltoit.Controlispassedfromthebootstraploaderdirectlytothekernelproper,toamodule called head.o for most architectures. It is an unfortunate historical artifact that both the bootstrap loaderandthekernelpropercontainamodulecalledhead.obecauseitisasourceofconfusiontothe new embedded Linux developer. The head.o module in the bootstrap loader might be more appropriately called kernel_bootstrap_loader_head.o, although I doubt that the kernel developers wouldacceptthispatch.Infact,arecentLinux2.6sourcetreecontainsnofewerthan37sourcefiles namedhead.S.Thisisanotherreasonwhyyouneedtoknowyourwayaroundthekernelsourcetree. ReferbacktoFigure5-3foragraphicalviewoftheflowofcontrol.Whenthebootstraploader hascompleteditsjob,controlispassedtothekernelproper'shead.o,andfromtheretostart_kernel() inmain.c. 5.2.1.KernelEntryPoint:head.o Theintentionofthekerneldeveloperswastokeepthearchitecture-specifichead.omodulevery generic, without any specific machine[39] dependencies. This module, derived from the assembly languagefilehead.S,islocatedat.../arch/<ARCH>/kernel/head.S,where<ARCH>is replacedby thegivenarchitecture.TheexamplesinthischapterarebasedontheARM/XScale,asyouhaveseen, with<ARCH>=arm. Thehead.omoduleperformsarchitecture-andoftenCPU-specificinitializationinpreparationfor the main body of the kernel. CPU-specific tasks are kept as generic as possible across processor families. Machine-specific initialization is performed elsewhere, as you will discover shortly. Amongotherlow-leveltasks,head.operformsthefollowingtasks: •Checksforvalidprocessorandarchitecture •Createsinitialpagetableentries •Enablestheprocessor'smemorymanagementunit(MMU) •Establisheslimitederrordetectionandreporting •Jumpstothestartofthekernelproper,main.c Thesefunctionscontainsomehiddencomplexities.Manynoviceembeddeddevelopershavetried to single-step through parts of this code, only to find that the debugger becomes hopelessly lost. Althoughadiscussionofthecomplexitiesofassemblylanguageandthehardwaredetailsofvirtual memory is beyond the scope of this book, a few things are worth noting about this complicated module. When control is first passed to the kernel's head.o from the bootstrap loader, the processor is operating in what we used to call real mode in x86 terminology. In effect, the logical address containedintheprocessor'sprogramcounter[40](oranyotherregister,forthatmatter)istheactual physical address driven onto the processor's electrical memory address pins. Soon after the processor's registers and kernel data structures are initialized to enable memory translation, the processor'smemorymanagementunit(MMU)isturnedon.Suddenly,theaddressspaceasseenby the processor is yanked from beneath it and replaced by an arbitrary virtual addressing scheme determinedbythekerneldevelopers.Thiscreatesacomplexitythatcanreallybeunderstoodonlyby adetailedanalysisofboththeassemblylanguageconstructsandlogicalflow,aswellasadetailed knowledgeoftheCPUanditshardwareaddresstranslationmechanism.Inshort,physicaladdresses are replaced by logical addresses the moment the MMU is enabled. That's why a debugger can't single-stepthroughthisportionofcodeaswithordinarycode. Thesecondpointworthnotingisthelimitedavailablemappingatthisearlystageofthekernel bootprocess.Manydevelopershavestumbledintothislimitationwhiletryingtomodifyhead.ofor their particular platform.[41] One such scenario might go like this. Let's say you have a hardware devicethatneedsafirmwareloadveryearlyinthebootcycle.Onepossiblesolutionistocompile the necessary firmware statically into the kernel image and then reference it via a pointer to downloadittoyourdevice.However,becauseofthelimitedmemorymappingdoneatthispoint,itis quite possible that your firmware image will exist beyond the range that has been mapped at this earlystageinthebootcycle.Whenyourcodeexecutes,itgeneratesapagefaultbecauseyouhave attempted to access a memory region for which no valid mapping has been created inside the processor.Worseyet,apagefaulthandlerhasnotyetbeeninstalledatthisearlystage,soallyouget isanunexplainedsystemcrash.Atthisearlystageinthebootcycle,youareprettymuchguaranteed nottohaveanyerrormessagestohelpyoufigureoutwhat'swrong. You are wise to consider delaying any custom hardware initialization until after the kernel has booted, if at all possible. In this manner, you can rely on the well-known device driver model for access to custom hardware instead of trying to customize the much more complicated assembly language startup code. Numerous undocumented techniques are used at this level. One common exampleofthisistoworkaroundhardwareerratathatmayormaynotbedocumented.Amuchhigher pricewillbepaidindevelopmenttime,cost,andcomplexityifyoumustmakechangestotheearly startupassemblylanguagecode.Hardwareandsoftwareengineersshoulddiscussthesefactsduring earlystagesofhardwaredevelopment,whenoftenaminorhardwarechangecanleadtosignificant savingsinsoftwaredevelopmenttime. It is important to recognize the constraints placed upon the developer in a virtual memory environment. Many experienced embedded developers have little or no experience in this environment,andthescenariopresentedearlierisbutonesmallexampleofthepitfallsthatawaitthe developernewtovirtualmemoryarchitectures.Nearlyallmodern32-bitandlargermicroprocessors havememory-managementhardwareusedtoimplementvirtualmemoryarchitectures.Oneofthemost significant advantages of virtual memory machines is that they help separate teams of developers write large complex applications, while protecting other software modules, and the kernel itself, fromprogrammingerrors. 5.2.2.KernelStartup:main.c The final task performed by the kernel's own head.o module is to pass control to the primary kernelstartupfilewritteninC.Wespendagoodportionoftherestofthischapteronthisimportant file. Foreacharchitecture,thereisadifferentsyntaxandmethodology,buteveryarchitecture'shead.o modulehasasimilarconstructforpassingcontroltothekernelproper.FortheARMarchitectureit looksassimpleasthis: bstart_kernel ForPowerPC,itlookssimilartothis: lisr4,start_kernel@h orir4,r4,start_kernel@l lisr3,MSR_KERNEL@h orir3,r3,MSR_KERNEL@l mtsprSRR0,r4 mtsprSRR1,r3 rfi Withoutgoingintodetailsofthespecificassemblylanguagesyntax,bothoftheseexamplesresult inthesamething.Controlispassedfromthekernel'sfirstobjectmodule(head.o)totheClanguage routinestart_kernel()locatedin.../init/main.c.Herethekernelbeginstodevelopalifeofitsown. Thefilemain.cshouldbestudiedcarefullybyanyoneseekingadeeperunderstandingoftheLinux kernel,whatcomponentsmakeitup,andhowtheyareinitializedand/orinstantiated.main.cdoesall thestartupworkfortheLinuxkernel,frominitializingthefirstkernelthreadallthewaytomountinga rootfilesystemandexecutingtheveryfirstuserspaceLinuxapplicationprogram. The function start_kernel() is by far the largest function in main.c. Most of the Linux kernel initializationtakesplaceinthisroutine.Ourpurposehereistohighlightthoseparticularelementsthat willproveuseful inthe contextofembeddedsystemsdevelopment.Itisworthrepeating:Studying main.cisagreatwaytospendyourtimeifyouwanttodevelopabetterunderstandingoftheLinux kernelasasystem. 5.2.3.ArchitectureSetup Amongthefirstfewthingsthathappenin.../init/main.cinthestart_kernel()functionisthecallto setup_arch().Thisfunctiontakesasingleparameter,apointertothekernelcommandlineintroduced earlieranddetailedinthenextsection. setup_arch(&command_line); This statement calls an architecture-specific setup routine responsible for performing initializationtaskscommonacrosseachmajorarchitecture.Amongotherfunctions,setup_arch()calls functions that identify the specific CPU and provides a mechanism for calling high-level CPUspecific initialization routines. One such function, called directly by setup_arch(), is setup_processor(), found in .../arch/arm/kernel/setup.c. This function verifies the CPU ID and revision,callsCPU-specificinitializationfunctions,anddisplaysseverallinesofinformationonthe consoleduringboot. An example of this output can be found in Listing 5-3, lines 3 through 8. Here you can see the CPUtype,IDstring,andrevisionreaddirectlyfromtheprocessorcore.Thisisfollowedbydetails oftheprocessorcachetypeandsize.Inthisexample,theIXP425hasa32KBI(instruction)cache and32KBD(data)cache,alongwithotherimplementationdetailsoftheinternalprocessorcache. One of the final actions of the architecture setup routines is to perform any machine-dependent initialization.Theexactmechanismforthisvariesacrossdifferentarchitectures.ForARM,youwill find machine-specific initialization in the .../arch/arm/mach-* series of directories, depending on your machine type. MIPS architecture also contains directories specific to supported reference platforms. For PowerPC, there is a machine-dependent structure that contains pointers to many commonsetupfunctions.WeexaminethisinmoredetailinChapter16,"PortingLinux." 5.3.KernelCommandLineProcessing Following the architecture setup, main.c performs generic early kernel initialization and then displaysthekernelcommandline.Line10ofListing5-3isreproducedhereforconvenience. Kernelcommandline:console=ttyS0,115200ip=bootproot=/dev/nfs Inthissimpleexample,thekernelbeingbootedisinstructedtoopenaconsoledeviceonserial port device ttyS0 (usually the first serial port) at a baud rate of 115Kbps. It is being instructed to obtainitsinitialIPaddressinformationfromaBOOTPserverandtomountarootfilesystemviathe NFSprotocol.(WecoverBOOTPlaterinChapter12,"EmbeddedDevelopmentEnvironment,"and NFSinChapters9,"FileSystems,"and12.Fornow,welimitthediscussiontothekernelcommand linemechanism.) Linuxistypicallylaunchedbyabootloader(orbootstraploader)withaseriesofparametersthat havecometobecalledthekernelcommandline.Althoughwedon'tactuallyinvokethekernelusing acommandpromptfromashell,manybootloaderscanpassparameterstothekernelinafashionthat resemblesthiswell-knownmodel.OnsomeplatformswhosebootloadersarenotLinuxaware,the kernelcommandlinecanbedefinedatcompiletimeandbecomeshardcodedaspartofthekernel binaryimage.Onotherplatforms(suchasadesktopPCrunningRedHatLinux),thecommandline canbemodifiedbytheuserwithouthavingtorecompilethekernel.Thebootstraploader(Grubor LilointhedesktopPCcase)buildsthekernelcommandlinefromaconfigurationfileandpassesitto the kernel during the boot process. These command line parameters are a boot mechanism to set initialconfigurationnecessaryforproperbootonagivenmachine. Numerous command line parameters are defined throughout the kernel. The .../Documentation subdirectory in the kernel source contains a file called kernel-parameters.txt containing a list of kernel command lineparameters in dictionaryorder.Rememberthepreviouswarningaboutkernel documentation:Thekernelchangesfarfasterthanthedocumentation.Usethisfileasaguide,butnot adefinitivereference.Morethan400distinctkernelcommandlineparametersaredocumentedinthis file,anditcannotbeconsideredacomprehensivelist.Forthat,youmustreferdirectlytothesource code. Thebasicsyntaxforkernelcommandlineparametersisfairlysimpleandmostlyevidentfromthe exampleinline10ofListing5-3.Kernelcommandlineparameterscanbeeitherasingletextword,a key=valuepair,orakey=value1,value2,….keyandmultivalueformat.Itisuptotheconsumerof this information to process the data as delivered. The command line is available globally and is processed by many modules as needed. As noted earlier, setup_arch() in main.c is called with the kernel command line as its only argument. This is to pass architecture-specific parameters and configurationdirectivestotherelevantportionsofarchitecture-andmachine-specificcode. Devicedriverwritersandkerneldeveloperscanaddadditionalkernelcommand-lineparameters fortheirownspecificneeds.Let'stakealookatthemechanism.Unfortunately,somecomplications areinvolvedinusingandprocessingkernelcommandlineparameters.Thefirstoftheseisthatthe originalmechanismisbeingdeprecatedinfavorofamuchmorerobustimplementation.Thesecond complication is that we need to comprehend the complexities of a linker script file to fully understandthemechanism.[42] It'snotnecessarilyallthatcomplex,butmostofusneverneedtounderstandalinkerscriptfile. Theembeddedengineerdoes.ItiswelldocumentedintheGNULDmanualreferencedattheendof thischapter. 5.3.1.The__setupMacro Asan exampleof theuseofkernelcommandlineparameters,considerthespecification ofthe console device. We want this device to be initialized early in the boot cycle so that we have a destinationforconsolemessagesduringboot.Thisinitializationtakesplaceinakernelobjectcalled printk.o.TheCsourcefile forthismoduleisfoundin.../kernel/printk.c.Theconsoleinitialization routine is called console_setup() and takes the kernel command line parameter string as its only argument. Thechallengeistocommunicatetheconsoleparametersspecifiedonthekernelcommandlineto thesetupanddevicedriverroutinesthatrequirethisdatainamodularandgeneralfashion.Further complicatingtheissueisthattypicallythecommandlineparametersarerequiredearly,before(orin timefor)thosemodulesthatneedthem.Thestartupcodeinmain.c,wherethemainprocessingofthe kernelcommandlinetakesplace,cannotpossiblyknowthedestinationfunctionsforeachofhundreds of kernel command line parameters without being hopelessly polluted with knowledge from every consumer of these parameters. What is needed is a flexible and generic way to pass these kernel commandlineparameterstotheirconsumers. In Linux 2.4 and earlier kernels, developers used a simple macro to generate a not-so-simple sequence of code. Although it is being deprecated, the __setup macro is still in widespread use throughoutthekernel.WenextusethekernelcommandlinefromListing5-3todemonstratehowthe __setupmacroworks. From the previous kernel command line (line 10 of Listing 5-3), this is the first complete commandlineparameterpassedtothekernel: console=ttyS0,115200 For the purposes of this example, the actual meaning of the parameters is irrelevant. Our goal hereistoillustratethemechanism,sodon'tbeconcernedifyoudon'tunderstandtheargumentorits values. Listing 5-4 is a snippet of code from .../kernel/printk.c. The body of the function has been strippedbecauseitisnotrelevanttothediscussion.ThemostrelevantpartofListing5-4isthelast line,theinvocationofthe__setupmacro.Thismacroexpectstwoarguments;inthiscase,itispassed astringliteralandafunctionpointer.Itisnocoincidencethatthestringliteralpassedtothe__setup macro is the same as the first eight characters of the kernel command line related to the console: console=. Listing5-4.ConsoleSetupCodeSnippet /* *Setupalistofconsoles.Calledfrominit/main.c */ staticint__initconsole_setup(char*str) { charname[sizeof(console_cmdline[0].name)]; char*s,*options; intidx; /* *Decodestrintoname,index,options. */ return1; } __setup("console=",console_setup); You can think of this macro as a registration function for the kernel command-line console parameter. In effect, it says: When the console= string is encountered on the kernel command line, invoke the function represented by the second __setup macro argumentin this case, the console_setup()function.Buthowisthisinformationcommunicatedtotheearlysetupcode,outside this module, which has no knowledge of the console functions? The mechanism is both clever and somewhatcomplicated,andreliesonlistsbuiltbythelinker. The details are hidden in a set of macros designed to conceal the syntactical tedium of adding sectionattributes(andotherattributes)toaportionofobjectcode.Theobjectiveistobuildastatic list of string literals associated with function pointers. This list is emitted by the compiler in a separately named ELF section in the final vmlinux ELF image. It is important to understand this technique;itisusedinseveralplaceswithinthekernelforspecial-purposeprocessing. Let'snowexaminehowthisisdoneforthe__setupmacrocase.Listing5-5isaportionofcode fromtheheaderfile.../include/linux/init.hdefiningthe__setupfamilyofmacros. Listing5-5.Familyof__setupMacroDefinitionsfrominit.h ... #define__setup_param(str,unique_id,fn,early)\ staticchar__setup_str_##unique_id[]__initdata=str;\ staticstructobs_kernel_param__setup_##unique_id\ __attribute_used__\ __attribute__((__section__(".init.setup")))\ __attribute__((aligned((sizeof(long)))))\ ={__setup_str_##unique_id,fn,early} #define__setup_null_param(str,unique_id)\ __setup_param(str,unique_id,NULL,0) #define__setup(str,fn\ __setup_param(str,fn,fn,0) ... Listing 5-5 is the author's definition of syntactical tedium! Recall from Listing 5-4 that our invocationoftheoriginal__setupmacrolookedlikethis: __setup("console=",console_setup); Withsomeslightsimplification,hereiswhatthecompiler'spreprocessorproducesaftermacro expansion: staticchar__setup_str_console_setup[]__initdata="console="; staticstructobs_kernel_param__setup_console_setup\ __attribute__((__section__(".init.setup")))={__setup_str_console_setup,console_setup,0}; Tomakethismorereadable,wehavesplitthesecondandthirdlines,asindicatedbytheUNIX line-continuationcharacter\. Wehaveintentionallyleftouttwocompilerattributeswhosedescriptiondoesnotaddanyinsight to this discussion. Briefly, the __attribute_used__ (itself a macro hiding further syntactical tedium) tellsthecompilertoemitthefunctionorvariable,eveniftheoptimizerdeterminesthatitisunused. [43]The__attribute__(aligned) tellsthecompilertoalign thestructuresonaspecificboundary,in thiscasesizeof(long). Whatwehaveleftaftersimplificationistheheartofthemechanism.First,thecompilergenerates anarrayofcharacterscalled__setup_str_console_setup[]initializedtocontainthestringconsole=. Next,thecompilergeneratesastructurethatcontainsthreemembers:apointertothekernelcommand linestring(thearrayjustdeclared),thepointertothesetupfunctionitself,andasimpleflag.Thekey tothemagichereisthesectionattributeattachedtothestructure.Thisattributeinstructsthecompiler toemitthisstructureintoaspecialsectionwithintheELFobjectmodule,called.init.setup.During thelink stage,allthestructuresdefined usingthe__setupmacroare collectedandplacedintothis .init .setup section, in effect creating an array of these structures. Listing 5-6, a snippet from .../init/main.c,showshowthisdataisaccessedandused. Listing5-6.KernelCommandLineProcessing 1externstructobs_kernel_param__setup_start[],__setup_end[]; 2 3staticint__initobsolete_checksetup(char*line) 4{ 5structobs_kernel_param*p; 6 7p=__setup_start; 8do{ 9intn=strlen(p->str); 10if(!strncmp(line,p->str,n)){ 11if(p->early){ 12/*Alreadydoneinparse_early_param?(Needs 13*exactmatchonparampart)*/ 14if(line[n]=='\0'||line[n]=='=') 15return1; 16}elseif(!p->setup_func){ 17printk(KERN_WARNING"Parameter%sisobsolete," 18"ignored\n",p->str); 19return1; 20}elseif(p->setup_func(line+n)) 21return1; 22} 23p++; 24}while(p<__setup_end); 25return0; 26} Examination of this code should be fairly straightforward, with a couple of explanations. The function is called with a single command line argument, parsed elsewhere within main.c. In the examplewe'vebeendiscussing,linewouldpointtothestringconsole=ttyS0,115200,whichisone component from the kernel command line. The two external structure pointers __setup_start and __setup_endaredefinedinalinkerscriptfile,notinaCsourceorheaderfile.Theselabelsmarkthe startandendofthearrayofobs_kernel_paramstructuresthatwereplacedinthe.init.setupsectionof theobjectfile. The code in Listing 5-6 scans all these structures via the pointer p to find a match for this particularkernelcommandlineparameter.Inthiscase,thecodeissearchingforthestringconsole= andfindsamatch.Fromtherelevantstructure,thefunctionpointerelementreturnsapointertothe console_setup()function,whichiscalledwiththebalanceoftheparameter(thestringttyS0,115200) asitsonlyargument.Thisprocessisrepeatedforeveryelementinthekernelcommandlineuntilthe kernelcommandlinehasbeencompletelyexhausted. Thetechniquejustdescribed,collectingobjectsintolistsinuniquelynamedELFsections,isused in many places in the kernel. Another example of this technique is the use of the __init family of macrostoplaceone-timeinitializationroutinesintoacommonsectionintheobjectfile.Itscousin __initdata,usedtomarkone-time-usedataitems,isusedbythe__setupmacro.Functionsanddata markedasinitializationusingthesemacrosarecollectedintoaspeciallynamedELFsection.Later, after these one-time initialization functions and data objects have been used, the kernel frees the memoryoccupiedbytheseitems.Youmighthaveseenthefamiliarkernelmessagenearthefinalpart of the boot process saying, "Freeing init memory: 296K." Your mileage may vary, but a third of a megabyteiswellworththeeffortofusingthe__initfamilyofmacros.Thisisexactlythepurposeof the__initdatamacrointheearlierdeclarationof__setup_str_console_setup[]. Youmighthavebeenwonderingabouttheuseofsymbolnamesprecededwithobsolete_.Thisis becausethekerneldevelopersarereplacingthekernelcommandlineprocessingmechanismwitha more generic mechanism for registering both boot time and loadable module parameters. At the present time, hundreds of parameters are declared with the __setup macro. However, new development is expected to use the family of functions defined by the kernel header file .../include/linux/moduleparam.h, most notably, the family of module_param* macros. These are explainedinmoredetailinChapter8,"DeviceDriverBasics,"whenweintroducedevicedrivers. Thenewmechanismmaintainsbackwardcompatibilitybyincludinganunknownfunctionpointer argument in the parsing routine. Thus, parameters that are unknown to the module_param* infrastructure are considered unknown, and the processing falls back to the old mechanism under control of the developer. This is easily understood by examining the well-written code in .../kernel/params.candtheparse_args()callsin.../init/main.c. The last point worth mentioning is the purpose of the flag member of the obs_kernel_param structurecreatedbythe__setupmacro.ExaminationofthecodeinListing5-6shouldmakeitclear. The flag in the structure, called early, is used to indicate whether this particular command line parameter was already consumed earlier in the boot process. Some command line parameters are intendedforconsumptionveryearlyinthebootprocess,andthisflagprovidesamechanismforan earlyparsingalgorithm.Youwillfindafunctioninmain.ccalleddo_early_param()thattraversesthe linker-generated array of __setup- generated structures and processes each one marked for early consumption.Thisgivesthedevelopersomecontroloverwheninthebootprocessthisprocessingis done. 5.4.SubsystemInitialization Many kernel subsystems are initialized by the code found in main.c. Some are initialized explicitly,aswiththecallstoinit_timers()andconsole_init(),whichneedtobecalledveryearly. Othersareinitializedusingatechniqueverysimilartothatdescribedearlierforthe__setupmacro. In short, the linker builds lists of function pointers to various initialization routines, and a simple loopisusedtoexecuteeachinturn.Listing5-7showshowthisworks. Listing5-7.ExampleInitializationRoutine staticint__initcustomize_machine(void){ /*customizesplatformdevices,oraddsnewones*/ if(init_machine)init_machine(); return0; } arch_initcall(customize_machine); This code snippet comes from .../arch/arm/kernel/setup.c. It is a simple routine designed to provideacustomizationhookforaparticularboard. 5.4.1.The*__initcallMacros NoticetwoimportantthingsabouttheinitializationroutineinListing5-7.First,itisdefinedwith the __init macro. As we saw earlier, this macro applies the section attribute to declare that this functiongetsplacedintoasectioncalled.init.textinthevmlinuxELFfile.Recallthatthepurposeof placingthisfunctionintoaspecialsectionoftheobjectfileissothememoryspacethatitoccupies canbereclaimedwhenitisnolongerneeded. The second thing to notice is the macro immediately following the definition of the function: arch_initcall(customize_machine). This macro is part of a family of macros defined in .../include/linux/init.h.ThesemacrosarereproducedhereasListing5-8. Listing5-8.initcallFamilyofMacros #define__define_initcall(level,fn)\ staticinitcall_t__initcall_##fn__attribute_used__\ __attribute__((__section__(".initcall"level".init")))=fn #definecore_initcall(fn)__define_initcall("1",fn) #definepostcore_initcall(fn)__define_initcall("2",fn) #definearch_initcall(fn)__define_initcall("3",fn) #definesubsys_initcall(fn)__define_initcall("4",fn) #definefs_initcall(fn)__define_initcall("5",fn) #definedevice_initcall(fn)__define_initcall("6",fn) #definelate_initcall(fn)__define_initcall("7",fn) Inasimilarfashiontothe__setupmacropreviouslydetailed,thesemacrosdeclareadataitem basedonthenameofthefunction,andusethesectionattributetoplacethisdataitemintoauniquely named section of the vmlinux ELF file. The benefit of this approach is that main.c can call an arbitraryinitializationfunctionforasubsystemthatithasnoknowledgeof.Theonlyotheroption,as mentionedearlier,istopollutemain.cwithknowledgeofeverysubsysteminthekernel. AsyoucanseefromListing5-8,thenameofthesectionis.initcallN.init,whereNisthelevel definedbetween1and7.Thedataitemisassignedtheaddressofthefunctionbeingnamedinthe macro.IntheexampledefinedbyListings5-7and5-8,thedataitemwouldbeasfollows(simplified byomittingthesectionattribute): staticinitcall_t__initcall_customize_machine=customize_machine; Thisdataitemisplacedinthekernel'sobjectfileinasectioncalled.initcall1.init. Thelevel(N)isusedtoprovideanorderingofinitializationcalls.Functionsdeclaredusingthe core_initcall() macro are called before all others. Functions declared using the postcore_initcall() macrosarecallednext,andsoon,whilethosedeclaredwithlate_initcall()arethelastinitialization functionstobecalled. In a fashion similar to the __setup macro, you can think of this family of *_initcall macros as registration functions for kernel subsystem initialization routines that need to be run once at kernel startupandthenneverusedagain.Thesemacrosprovideamechanismforcausingtheinitialization routinetobeexecutedduringsystemstartup,andamechanismtodiscardthecodeandreclaimthe memory after the routine has been executed. The developer is also provided up to seven levels of whentoperformtheinitializationroutines.Therefore,ifyouhaveasubsystemthatreliesonanother beingavailable,youcanenforcethisorderingusingtheselevels.Ifyougrepthekernelforthestring [a-z]*_initcall,youwillseethatthisfamilyofmacrosisusedextensively. Onefinalnoteaboutthe*_initcallfamilyofmacros:Theuseofmultiplelevelswasintroduced duringthedevelopmentofthe2.6kernelseries.Earlierkernelversionsusedthe__initcall()macro for this purpose. This macro is still in widespread use, especially in device drivers. To maintain backwardcompatibility,thismacrohasbeendefinedtodevice_initcall(),whichhasbeendefinedas alevel6initcall. 5.5.TheinitThread Thecodefoundin.../init/main.cisresponsibleforbringingthekerneltolife.Afterstart_kernel() performssomebasickernelinitialization,callingearlyinitializationfunctionsexplicitlybyname,the veryfirstkernelthreadisspawned.Thisthreadeventuallybecomesthekernelthreadcalledinit(), withaprocessid(PID)of1.Asyouwilllearn,init()becomestheparentofallLinuxprocessesin userspace.Atthispointinthebootsequence,twodistinctthreadsarerunning:thatrepresentedby start_kernel()andnowinit().Theformergoesontobecometheidleprocess,havingcompletedits work.Thelatterbecomestheinitprocess.ThiscanbeseeninListing5-9. Listing5-9.CreationofKernelinitTHRead staticvoidnoinlinerest_init(void)__releases(kernel_lock){ kernel_thread(init,NULL,CLONE_FS|CLONE_SIGHAND); numa_default_policy(); unlock_kernel(); preempt_enable_no_resched(); /* *Thebootidlethreadmustexecuteschedule() *atleastonetogetthingsmoving: */ schedule(); cpu_idle(); } Thestart_kernel()functioncallsrest_init(),reproducedinListing5-9.Thekernel'sinitprocessis spawnedbythecalltokernel_thread().initgoesontocompletetherestofthesysteminitialization, whilethethreadofexecutionstartedbystart_kernel()loopsforeverinthecalltocpu_idle(). Thereasonforthisstructureisinteresting.Youmighthavenoticedthatstart_kernel(),arelatively large function, was marked with the __init macro. This means that the memory it occupies will be reclaimedduringthefinalstagesofkernelinitialization.Itisnecessarytoexitthisfunctionandthe address space that it occupies before reclaiming its memory. The answer to this was for start_kernel()tocallrest_init(),showninListing5-9,amuchsmallerpieceofmemorythatbecomes theidleprocess. 5.5.1.Initializationviainitcalls Wheninit()isspawned,iteventuallycallsdo_initcalls(),whichisthefunctionresponsiblefor calling all the initialization functions registered with the *_initcall family of macros. The code is reproducedinListing5-10insimplifiedform. Listing5-10.Initializationviainitcalls staticvoid__initdo_initcalls(void){ initcall_t*call; for(call=&__initcall_start;call<&__initcall_end;call++){ if(initcall_debug){ printk(KERN_DEBUG"Callinginitcall0x%p",*call); print_symbol(":%s()",(unsignedlong)*call); printk("\n"); } (*call)(); } } This code is self-explanatory, except for the two labels marking the loop boundaries: __initcall_startand__initcall_end.TheselabelsarenotfoundinanyCsourceorheaderfile.They are defined in the linker script file used during the link stage of vmlinux. These labels mark the beginning and end of the list of initialization functions populated using the *_initcall family of macros. You can see each of the labels by looking at the System.map file in the top-level kernel directory.Theyallbeginwiththestring__initcall,asdescribedinListing5-8. In case you were wondering about the debug print statements in do_initcalls(), you can watch these calls being executed during bootup by setting the kernel command line parameter initcall_debug.Thiscommandlineparameterenablestheprintingofthedebuginformationshownin Listing 5-10. Simply start your kernel with the kernel command line parameter initcall_debug to enablethisdiagnosticoutput.[44] Hereisanexampleofwhatyouwillseewhenyouenablethesedebugstatements: ... Callinginitcall0xc00168f4:tty_class_init+0x0/0x3c() Callinginitcall0xc000c32c:customize_machine+0x0/0x2c() Callinginitcall0xc000c4f0:topology_init+0x0/0x24() Callinginitcall0xc000e8f4:coyote_pci_init+0x0/0x20() PCI:IXP4xxishost PCI:IXP4xxUsingdirectaccessformemoryspace ... Noticethecalltocustomize_machine(),theexampleofListing5-7.Thedebugoutputincludesthe virtual kernel address of the function (0xc000c32c, in this case) and the size of the function (0x2c here.)Thisis ausefulway to seethedetailsofkernelinitialization,especiallytheorderinwhich varioussubsystemsandmodulesgetcalled.Evenonamodestlyconfiguredembeddedsystem,dozens of these initialization functions are invoked in this manner. In this example taken from an ARM XScaleembeddedtarget,thereare92suchcallstovariouskernel-initializationroutines. 5.5.2.FinalBootSteps Having spawned the init() thread and all the various initialization calls have completed, the kernelperformsitsfinalstepsinthebootsequence.Theseincludefreeingthememoryusedbythe initialization functions and data, opening a system console device, and starting the first userspace process.Listing5-11reproducesthelaststepsinthekernel'sinit()frommain.c. Listing5-11.FinalKernelBootStepsfrommain.c if(execute_command){ run_init_process(execute_command); printk(KERN_WARNING"Failedtoexecute%s.Attemptingdefaults...\n",execute_command); } run_init_process("/sbin/init"); run_init_process("/etc/init"); run_init_process("/bin/init"); run_init_process("/bin/sh"); panic("Noinitfound.Trypassinginit=optiontokernel."); Noticethatifthecodeproceedstotheendoftheinit()function,akernelpanicresults.Ifyou've spentanytimeexperimentingwithembeddedsystemsorcustomrootfilesystems,you'veundoubtedly encounteredthisverycommonerrormessageasthelastlineofoutputonyourconsole.Itisoneofthe mostfrequentlyaskedquestions(FAQs)onavarietyofpublicforumsrelatedtoLinuxandembedded systems. Onewayoranother,oneoftheserun_init_process()commandsmustproceedwithouterror.The run_init_process()functiondoesnotreturnonsuccessfulinvocation.Itoverwritesthecallingprocess with the new one, effectively replacing the current process with the new one. It uses the familiar execve()systemcallforthisfunctionality.Themostcommonsystemconfigurationsspawn/sbin/init astheuserland[45]initializationprocess.Westudythisfunctionalityindepthinthenextchapter. Oneoptionavailabletotheembeddedsystemdeveloperistouseacustomuserlandinitialization program. That is the purpose of the conditional statement in the previous code snippet. If execute_commandisnon-null,itpointstoastringcontainingacustomuser-suppliedcommandtobe executedinuserspace.Thedeveloperspecifiesthiscommandonthekernelcommandline,anditis set via the __setup macro we examined earlier in this chapter. An example kernel command line incorporatingseveralconceptsdiscussedinthischaptermightlooklikethis: initcall_debuginit=/sbin/myinitconsole=ttyS1,115200root=/dev/hda1 This kernel command line instructs the kernel to display all the initialization routines as encountered,configurestheinitialconsoledeviceas/dev/ttyS1at115kbps,andexecutesacustom userspaceinitializationprocesscalledmyinit,locatedinthe/sbindirectoryontherootfilesystem.It directsthekerneltomountitsrootfilesystemfromthedevice/dev/hda1,whichisthefirstIDEhard drive.Notethat,ingeneral,theorderofparametersgivenonthekernelcommandlineisirrelevant. Thenextchaptercoversthedetailsofuserspacesysteminitialization. 5.6.ChapterSummary •TheLinuxkernelprojectislargeandcomplex.Understandingthestructureandcompositionof thefinalimageiskeytolearninghowtocustomizeyourownembeddedproject. •Manyarchitecturesconcatenateanarchitecture-specificbootstraploaderontothekernelbinary image to set up the proper execution environment required by the Linux kernel. We presented the bootstraploaderbuildstepstodifferentiatethisfunctionalityfromthekernelproper. •UnderstandingtheinitializationflowofcontrolwillhelpdeepenyourknowledgeoftheLinux kernelandprovideinsightintohowtocustomizeforyourparticularsetofrequirements. •Wefoundthekernelentrypointinhead.oandfollowedtheflowofcontrolintothefirstkernelC file,main.c.Welookedatabootingsystemandthemessagesitproduced,alongwithanoverviewof manyoftheimportantinitializationconcepts. • The kernel command line processing and the mechanisms used to declare and process kernel command line parameters was presented. This included a detailed look at some advanced coding techniquesforcallingarbitraryunknownsetuproutinesusinglinker-producedtables. • The final kernel boots steps produce the first userspace processes. Understanding this mechanism and its options will enable you to customize and troubleshoot embedded Linux startup issues. 5.6.1.SuggestionsforAdditionalReading GNUCompilerCollectiondocumentation: http://gcc.gnu.org/onlinedocs/gcc[46] UsingLD,theGNUlinker http://www.gnu.org/software/binutils/manual/ld-2.9.1/ld.html Kerneldocumentation: .../Documentation/kernel-parameters.txt Chapter6.SystemInitialization InChapter2,"YourFirstEmbeddedExperience,"wepointedoutthattheLinuxkernelitselfisbut asmallpartofanyembeddedLinuxsystem.Afterthekernelhasinitializeditself,itmustmountaroot filesystemandexecuteasetofdeveloper-definedinitializationroutines.Inthischapter,weexamine thedetailsofpost-kernelsysteminitialization. Webeginbylookingattherootfilesystemanditslayout.Nextwedevelopandstudyaminimal systemconfiguration.Laterinthischapter,weaddfunctionalitytotheminimalsystemconfiguration to produce useful example embedded system configurations. We complete the coverage of system initialization by introducing the initial ramdisk, or initrd, and its operation and use. The chapter concludeswithabrieflookatLinuxshutdownlogic. 6.1.RootFileSystem In Chapter 5, "Kernel Initialization," we examined the Linux kernel's behavior during the initializationprocess.Wemadeseveralreferencestomountingarootfilesystem.Linux,likemany otheradvancedoperatingsystems,requiresarootfilesystemtorealizethebenefitsofitsservices. AlthoughitiscertainlypossibletouseLinuxinanenvironmentwithoutafilesystem,itmakeslittle sensebecausemostofthefeaturesandvalueofLinuxwouldbelost.Itwouldbesimilartoputting yourentiresystemapplicationintoanoverbloateddevicedriverorkernelthread. The root file system refers to the file system mounted at the base of the file system hierarchy, designatedsimplyas/.AsyouwilldiscoverinChapter9,"FileSystems,"evenasmallembedded Linuxsystemtypicallymountsseveralfilesystemsondifferentlocationsinthefilesystemhierarchy. The proc file system, introduced in Chapter 9, is an example. It is a special-purpose file system mounted at /proc under the root file system. The root file system is simply the first file system mountedatthebaseofthefilesystemhierarchy. Asyouwillshortlysee,therootfilesystemhasspecialrequirementsforaLinuxsystem.Linux expectstherootfilesystemtocontainprogramsandutilitiestobootasystem,initializeservicessuch asnetworkingandasystemconsole,loaddevicedrivers,andmountadditionalfilesystems. 6.1.1.FHS:FileSystemHierarchyStandard SeveralkerneldevelopersauthoredastandardgoverningtheorganizationandlayoutofaUNIX file system. The File System Hierarchy Standard (FHS) establishes a minimum baseline of compatibility between Linux distributions and application programs. You'll find a reference to this standard in Section 6.7.1 "Suggestions for Additional Reading" at the end of this chapter. You are encouragedtoreviewtheFHSstandardforabetterbackgroundonthelayoutandrationaleofUNIX filesystemorganization. Many Linux distributions have directory layouts closely matching that described in the FHS standard.ThestandardexiststoprovideoneelementofacommonbasebetweendifferentUNIXand Linuxdistributions.TheFHSstandardallowsyourapplicationsoftware(anddevelopers)topredict wherecertainsystemelements,includingfilesanddirectories,canbefoundonthefilesystem. 6.1.2.FileSystemLayout Where space is a concern, many embedded systems developers create a very small root file system on a bootable device (such as Flash memory) and later mount a larger file system from anotherdevice,perhapsaharddiskornetworkNFSserver.Infact,itisnotuncommontomounta largerrootfilesystemrightontopoftheoriginalsmallone.You'llseeanexampleofthatwhenwe examinetheinitialramdisk(initrd)laterinthischapter. AsimpleLinuxrootfilesystemmightcontainthefollowingtop-leveldirectoryentries: . | |--bin |--dev |--etc |--lib |--sbin |--usr |--var |--tmp Table6-1detailsthemostcommoncontentsofeachoftheserootdirectoryentries. Table6-1.Top-LevelDirectories Directory Contents bin Binaryexecutables,usablebyallusersonthesystem[47] dev Devicenodes(seeChapter8,"DeviceDriverBasics") etc Localsystem-configurationfiles lib Systemlibraries,suchasthestandardClibraryandmanyothers sbin Binaryexecutablesusuallyreservedforsuperuseraccountsonthesystem usr Asecondaryfilesystemhierarchyforapplicationprograms,usuallyread-only var Containsvariablefiles,suchassystemlogsandtemporaryconfigurationfiles tmp Temporaryfiles TheverytopoftheLinuxfilesystemhierarchyisreferencedbytheforwardslashcharacter(/)by itself.Forexample,tolistthecontentsoftherootdirectory,onewouldtypethis: $ls/ Thisproducesalistingsimilartothefollowing: root@coyote:/#ls/ bindevetchomelibmntoptprocrootsbintmpusrvar root@coyote:/# This directory listing contains directory entries for additional functionality, including /mnt and /proc.Noticethatwereferencethesedirectoryentriesprecededbytheforwardslash,indicatingthat thepathtothesetop-leveldirectoriesstartsfromtherootdirectory. 6.1.3.MinimalFileSystem Toillustratetherequirementsoftherootfilesystem,wehavecreatedaminimalrootfilesystem. This example was produced on the ADI Engineering Coyote Reference board using an XScale processor.Listing6-1istheoutputfromtheTReecommandonthisminimalrootfilesystem. Listing6-1.ContentsofMinimalRootFileSystem . |--bin | ||--busybox | |'--sh->busybox |--dev | |'--console |--etc | |'--init.d | |'--rcS '--lib |--ld-2.3.2.so |--ld-linux.so.2->ld-2.3.2.so |--libc-2.3.2.so '--libc.so.6->libc-2.3.2.so 5directories,8files Thisrootconfigurationmakesuseofbusybox,apopularandaptlynamedtoolkitforembedded systems. In short, busybox is a stand-alone binary that provides support for many common Linux command line utilities. busybox is so pertinent for embedded systems that we devote Chapter 11, "BusyBox,"tothisflexibleutility. NoticeinourexampleminimumfilesysteminListing6-1thatthereareonlyeightfilesinfive directories.Thistinyrootfilesystembootsandprovidestheuserwithafullyfunctionalcommand promptontheserialconsole.Anycommandsthathavebeenenabledinbusybox[48]areavailableto theuser. Starting from /bin, we have the busybox executable and a soft link called sh pointing back to busybox.Youwillseeshortlywhythisisnecessary.Thefilein/devisadevicenode[49]requiredto openaconsoledeviceforinputandoutput.Althoughitisnotstrictlynecessary,thercSfileinthe /etc/init.ddirectoryisthedefaultinitializationscriptprocessedbybusyboxonstartup.IncludingrcS silencesthewarningmessageissuedbybusyboxifrcSismissing. The final directory entry and set of files required are the two libraries, GLIBC (libc-2.3.2.so) andtheLinuxdynamicloader(ld-2.3.2.so).GLIBCcontainsthestandardClibraryfunctions,suchas printf() and many others that most application programs depend on. The Linux dynamic loader is responsible for loading the binary executable into memory and performing the dynamic linking required by the application's reference to shared library functions. Two additional soft links are included, ld-linux.so.2 pointing back to ld-2.3.2.so and libc.so.6 referencing libc-2.3.2.so. These links provide version immunity and backward compatibility for the libraries themselves, and are foundonallLinuxsystems. Thissimpleroot file systemproducesafullyfunctionalsystem.OntheARM/XScaleboardon whichthiswastested,thesizeofthissmallrootfilesystemwasabout1.7MB.Itisinterestingtonote thatmorethan80percentofthatsizeiscontainedwithintheClibraryitself.Ifyouneedtoreduceits size for your embedded system, you might want to investigate the Library Optimizer Tool at http://libraryopt.sourceforge.net/. 6.1.4.TheRootFSChallenge The challenge of a root file system for an embedded device is simple to explain. It is not so simple to overcome. Unless you are lucky enough to be developing an embedded system with a reasonably large hard drive or large Flash storage on board, you might find it difficult to fit your applicationsandutilitiesontoasingleFlashmemorydevice.Althoughcostscontinuetocomedown for Flash storage, there will always be competitive pressure to reduce costs and speed time to market.OneofthesinglelargestreasonsLinuxcontinuestogrowinpopularityasanembeddedOSis thehugeandgrowingbodyofLinuxapplicationsoftware. Trimmingarootfilesystemtofitintoagivenstoragespacerequirementcanbedaunting.Many packagesandsubsystemsconsistofdozensorevenhundredsoffiles.Inadditiontotheapplication itself, many packages include configuration files, libraries, configuration utilities, icons, documentationfiles,localefilesrelatedtointernationalization,databasefiles,andmore.TheApache webserverfromtheApacheSoftwareFoundationisanexampleofapopularapplicationoftenfound in embedded systems. The base Apache package from one popular embedded Linux distribution contains254differentfiles.Furthermore,theyaren'tallsimplycopiedintoasingledirectoryonyour filesystem.TheyneedtobepopulatedinseveraldifferentlocationsonthefilesystemfortheApache applicationtofunctionwithoutmodification. Theseconceptsaresomeofthefundamentalaspectsofdistributionengineering,andtheycanbe quite tedious. Linux distribution companies such as Red Hat (in the desktop and enterprise market segments) and Monta Vista Software (in the embedded market segment) spend considerable engineeringresourcesonjustthis:packagingacollectionofprograms,libraries,tools,utilities,and applications that together make up a Linux distribution. By necessity, building a root file system employselementsofdistributionengineeringonasmallerscale. 6.1.5.Trial-and-ErrorMethod Untilrecently,theonlywaytopopulatethecontentsofyourrootfilesystemwastousethetrialand-errormethod.Perhapstheprocesscanbeautomatedbycreatingasetofscriptsforthispurpose, but the knowledge of which files are required for a given functionality still had to come from the developer.ToolssuchasRedHatPackageManager(rpm)canbeusedtoinstallpackagesonyour rootfilesystem.rpmhasreasonabledependencyresolutionwithingivenpackages,butitiscomplex andinvolvesa steep learning curve.Furthermore,usingrpmdoesnotlenditselfeasilytobuilding small root file systems because it has limited capability to strip unnecessary files from the installation,suchasdocumentationandunusedutilitiesinagivenpackage. 6.1.6.AutomatedFileSystemBuildTools The leading vendors of embedded Linux distributions ship very capable tools designed to automate the task of building root file systems in Flash or other devices. These tools are usually graphicalinnature,enablingthedevelopertoselectfilesbyapplicationorfunctionality.Theyhave features to strip unnecessary files such as documentation and other unneeded files from a package, andmanyhavethecapabilitytoselectattheindividualfilelevel.Thesetoolscanproduceavariety offilesystemformatsforlaterinstallationonyourchoiceofdevice.Contactyourfavoriteembedded Linuxdistributionvendorfordetailsonthesepowerfultools. 6.2.Kernel'sLastBootSteps In the previous chapter, we introduced the steps the kernel takes in the final phases of system boot.Thefinalsnippetofcodefrom.../init/main.cisreproducedinListing6-2forconvenience. Listing6-2.FinalBootStepsfrommain.c ... if(execute_command){ run_init_process(execute_command); printk(KERN_WARNING"Failedtoexecute%s.Attemptingdefaults...\n",execute_command); } run_init_process("/sbin/init"); run_init_process("/etc/init"); run_init_process("/bin/init"); run_init_process("/bin/sh"); panic("Noinitfound.Trypassinginit=optiontokernel."); Thisisthefinalsequenceofeventsforthekernelthreadcalledinitspawnedbythekernelduring the final stages of boot. The run_init_process() is a small wrapper around the execve() function, whichisakernelsystemcallwitharatherinterestingbehavior.Theexecve()functionneverreturns ifnoerrorconditionsareencounteredinthecall.Thememoryspaceinwhichthecallingthreadis executingisoverwrittenbythecalledprogram'smemoryimage.Ineffect,thecalledprogramdirectly replacesthecallingthread,includinginheritingitsProcessID(PID). The structure of this initialization sequence has been unchanged for a long time in the developmentoftheLinuxkernel.Infact,Linuxversion1.0containedsimilarconstructs.Essentially, thisisthestartofuserspace[50]processing.AsyoucanseefromListing6-2,unlesstheLinuxkernel issuccessfulinexecutingoneoftheseprocesses,thekernelwillhalt,displayingthemessagepassed inthepanic()systemcall.Ifyouhavebeenworkingwithembeddedsystemsforanylengthoftime, andespeciallyifyouhaveexperienceworkingonrootfilesystems,youaremorethanfamiliarwith thiskernelpanic()anditsmessage!IfyousearchonGoogleforthispanic()errormessage,youwill findpageafterpageofhitsforthisFAQ.Whenyoucompletethischapter,youwillbeanexpertat troubleshootingthiscommonfailure. Noticeakeyingredientoftheseprocesses:Theyareallprogramsthatareexpectedtoresideona rootfilesystemthathasasimilarstructuretothatpresentedinListing6-1.Thereforeweknowthat wemustatleastsatisfythekernel'srequirementforaninitprocessthatiscapableofexecutingwithin itsownenvironment. InlookingatListing6-2,thismeansthatatleastoneoftherun_init_process()functioncallsmust succeed.Youcanseethatthekerneltriestoexecuteoneoffourprogramsintheorderinwhichthey areencountered.Asyoucanseefromthelisting,ifnoneofthesefourprogramssucceeds,thebooting kernelissuesthedreadedpanic()functioncallanddiesrightthere.Remember,thissnippetofcode from.../init/main.cisexecutedonlyonceonbootup.Ifitdoesnotsucceed,thekernelcandolittlebut complainandhalt,whichitdoesthroughthepanic()functioncall. 6.2.1.FirstUserSpaceProgram OnmostLinuxsystems,/sbin/initisspawnedbythekernelonboot.Thisiswhyitisattempted firstfromListing6-2.Effectively,thisbecomesthefirstuserspaceprogramtorun.Toreview,thisis thesequence: 1.Mounttherootfilesystem 2.Spawnthefirstuserspaceprogram,which,inthisdiscussion,becomesinit InourexampleminimalrootfilesystemfromListing6-2,thefirstthreeattemptsatspawninga userspaceprocesswouldfailbecausewedidnotprovideanexecutablefilecalledinitanywhereon thefilesystem.RecallfromListing6-1thatwehadasoftlinkcalledshthatpointedbacktobusybox. Youshouldnowrealizethepurposeforthatsoftlink:Itcausesbusyboxtobeexecutedbythekernel as the initial process, while also satisfying the common requirement for a shell executable from userspace.[51] 6.2.2.ResolvingDependencies Itisnotsufficienttosimplyincludeanexecutablesuchasinitonyourfilesystemandexpectitto boot.Foreveryprocessyouplaceonyourrootfilesystem,youmustalsosatisfyitsdependencies. Most processes have two categories of dependencies: those that are needed to resolve unresolved references within a dynamically linked executable, and external configuration or data files that an applicationmightneed.Wehaveatooltofindtheformer,butthelattercanbesuppliedonlybyat leastacursoryunderstandingoftheapplicationinquestion. Anexamplewillhelpmakethisclear.Theinitprocessisadynamicallylinkedexecutable.Torun init,weneedtosatisfyitslibrarydependencies.Atoolhasbeendevelopedforthispurpose:ldd.To understandwhatlibrariesagivenapplicationrequires,simplyrunyourcross-versionoflddonthe binary: $ppc_4xxFP-lddinit libc.so.6=>/opt/eldk/ppc_4xxFP/lib/libc.so.6 ld.so.1=>/opt/eldk/ppc_4xxFP/lib/ld.so.1 $ Fromthislddoutput,wecanseethatthePowerPCinitexecutableinthisexampleisdependenton twolibraries.ThesearethestandardClibrary(libc.so.6)andtheLinuxdynamicloader(ld.so.1). Tosatisfythesecondcategoryofdependenciesforanexecutable,theconfigurationanddatafiles thatitmightneed,thereislittlesubstituteforsomeknowledgeabouthowthesubsystemworks.For example,initexpectstoreaditsoperationalconfigurationfromadatafilecalledinittablocatedon /etc. Unless you are using a tool that has this knowledge built in, such as those described in the earlierSection6.1.6,"AutomatedFileSystemBuildTools,"youmustsupplythatknowledge. 6.2.3.CustomizedInitialProcess Itisworthnotingthatthedevelopercancontrolwhichinitialprocessisexecutedatstartup.This isdonebyakernelcommandlineparameter.ItishintedatinListing6-2bythetextcontainedwithin thepanic()functioncall.BuildingonourkernelcommandlinefromChapter5,hereishowitmight lookwithadeveloper-specifiedinitprocess: console=ttyS0,115200ip=bootproot=/dev/nfsinit=/sbin/myinit Specifyinginit=inthekernelcommandlineinthisway,youmustprovideabinaryexecutableon your root file system in the /sbin directory called myinit. This would be the first process to gain controlatthecompletionofthekernel'sbootprocess. 6.3.TheInitProcess Unless you are doing something highly unusual, you will never need to provide a customized initial process because the capabilities of the standard init process are very flexible. The init program, together with a family of startup scripts that we examine shortly, implement what is commonly calledSystemV Init, from the original UNIX System V that used this schema. We now examinethispowerfulsystemconfigurationandcontrolutility. Wesawintheprevioussectionthatinitisthefirstuserspaceprocessspawnedbythekernelafter completion of the boot process. As you will learn, every process in a running Linux system has a child-parentrelationshipwithanotherprocessrunninginthesystem.initistheultimateparentofall user space processes in a Linux system. Furthermore, init provides the default set of environment parametersforallotherprocessestoinherit,includingsuchthingsasPATHandCONSOLE. Itsprimaryroleistospawnadditionalprocessesunderthedirectionofaspecialconfiguration file. This configuration file is usually stored as /etc/inittab. init has the concept of a runlevel. A runlevel can be thought of as a system state. Each runlevel is defined by the services enabled and programsspawneduponentrytothatrunlevel. initcanexistinasinglerunlevelatanygiventime.Runlevelsusedbyinitincluderunlevelsfrom 0to6andaspecialrunlevelcalledS.Runlevel0instructsinittohaltthesystem,whilerunlevel6 results in a system reboot. For each run-level, a set of startup and shutdown scripts is usually providedthatdefinetheactionasystemshouldtakeforeachrunlevel.Actionstoperformforagiven runlevelaredeterminedbythe/etc/inittabconfigurationfile,describedshortly. Severaloftherunlevelshavebeenreservedforspecificpurposesinmanydistributions.Table62detailstherunlevelsandtheirpurposeincommonuseinmanyLinuxdistributions. Table6-2.Runlevels Runlevel Purpose 0 Systemshutdown(halt) 1 Single-usersystemconfigurationformaintenance 2 Userdefined 3 Generalpurposemultiuserconfiguration 4 Userdefined 5 Multiuserwithgraphicaluserinterfaceonstartup 6 Systemrestart(reboot) Therunlevelscriptsarecommonlyfoundunderadirectorycalled/etc/rc.d/init.d.Hereyouwill find most of the scripts that enable and disable individual services. Services can be configured manually,byinvokingthescriptandpassingoneoftheappropriateargumentstothescript,suchas start,stop,orrestart.Listing6-3displaysanexampleofrestartingthenfsservice. Listing6-3.NFSRestart $/etc/rc.d/init.d/nfsrestart ShuttingdownNFSmountd:[OK] ShuttingdownNFSdaemon:[OK] ShuttingdownNFSquotas:[OK] ShuttingdownNFSservices:[OK] StartingNFSservices:[OK] StartingNFSquotas:[OK] StartingNFSdaemon:[OK] StartingNFSmountd:[OK] IfyouhavespentanytimewithadesktopLinuxdistributionsuchasRedHatorFedora,youhave undoubtedlyseenlineslikethisduringsystemstartup. Arunlevelisdefinedbytheservicesthatareenabledatthatrunlevel.MostLinuxdistributions contain a directory structure under /etc that contains symbolic links to the service scripts in /etc/rc.d/init.d.Theserunleveldirectoriesaretypicallyrootedat/etc/rc.d.Underthisdirectory,you will find a series of runlevel directories that contain startup and shutdown specifications for each runlevel.initsimplyexecutesthesescriptsuponentryandexitfromarunlevel.Thescriptsdefinethe systemstate,andinittabinstructsinitonwhichscriptstoassociatewithagivenrunlevel.Listing6-4 contains the directory structure beneath /etc/rc.d that drives the runlevel startup and shutdown behavioruponentrytoorexitfromthespecifiedrunlevel,respectively. Listing6-4.RunlevelDirectoryStructure $ls-l/etc/rc.d total96 drwxr-xr-x2rootroot4096Oct2010:19init.d -rwxr-xr-x1rootroot2352Mar162004rc drwxr-xr-x2rootroot4096Mar222005rc0.d drwxr-xr-x2rootroot4096Mar222005rc1.d drwxr-xr-x2rootroot4096Mar222005rc2.d drwxr-xr-x2rootroot4096Mar222005rc3.d drwxr-xr-x2rootroot4096Mar222005rc4.d drwxr-xr-x2rootroot4096Mar222005rc5.d drwxr-xr-x2rootroot4096Mar222005rc6.d -rwxr-xr-x1rootroot943Dec3116:36rc.local -rwxr-xr-x1rootroot25509Jan112005rc.sysinit EachoftherunlevelsisdefinedbythescriptscontainedinthercN.d,whereNistherunlevel. InsideeachrcN.ddirectory,youwillfindnumeroussymlinksarrangedinaspecificorder.These symboliclinksstartwitheitheraKoranS.ThosebeginningwithSpointtoservicescripts,which areinvokedwithstartupinstructions;thosestartingwithaKpointtoservicescriptsthatareinvoked withshutdowninstructions.AnexamplewithaverysmallnumberofservicesmightlooklikeListing 6-5. Listing6-5.ExampleRunlevelDirectory lrwxrwxrwx1rootroot17Nov252004S10network->../init.d/network lrwxrwxrwx1rootroot16Nov252004S12syslog->../init.d/syslog lrwxrwxrwx1rootroot16Nov252004S56xinetd->../init.d/xinetd lrwxrwxrwx1rootroot16Nov252004K50xinetd->../init.d/xinetd lrwxrwxrwx1rootroot16Nov252004K88syslog->../init.d/syslog lrwxrwxrwx1rootroot17Nov252004K90network->../init.d/network In this example, we are instructing the startup scripts to start three services upon entry to this fictitiousrunlevel: network,syslog,andxinetd. BecausetheS* scriptsare orderedwithanumeric tag,theywillbestartedinthisorder.Inasimilarfashion,whenexitingthisrunlevel,threeservices will be terminated: xinetd, syslog, and network. In a similar fashion, these services will be terminatedintheorderpresentedbythetwo-digitnumberfollowingtheKinthesymlinkfilename.In anactualsystem,therewouldundoubtedlybemanymoreentries.Youcanincludeyourownentries foryourowncustomapplications,too. Thetop-levelscriptthatexecutestheseservicestartupandshutdownscriptsisdefinedintheinit configurationfile,whichwenowexamine. 6.3.1.inittab Wheninitisstarted,itreadsthesystemconfigurationfile/etc/inittab.Thisfilecontainsdirectives foreachrunlevel,aswellasdirectivesthatapplytoallrun-levels.Thisfileandinit'sbehaviorare well documented in man pages on most Linux workstations, as well as by several books covering system administration. We do not attempt to duplicate those works; we focus on how a developer might configure inittab for an embedded system. For a detailed explanation of how inittab and init worktogether,viewthemanpageonmostLinuxworkstationsbytypingmaninitandmaninittab. Let'stakealookatatypicalinittabforasimpleembeddedsystem.Listing6-6containsasimple inittabexampleforasystemthatsupportsasinglerunlevelaswellasshutdownandreboot. Listing6-6.SimpleExampleinittab #/etc/inittab #Thedefaultrunlevel(2inthisexample) id:2:initdefault: #Thisisthefirstprocess(actuallyascript)toberun. si::sysinit:/etc/rc.sysinit #Executeourshutdownscriptonentrytorunlevel0 l0:0:wait:/etc/init.d/sys.shutdown #Executeournormalstartupscriptonenteringrunlevel2 l2:2:wait:/etc/init.d/runlvl2.startup #Thislineexecutesarebootscript(runlevel6) l6:6:wait:/etc/init.d/sys.reboot #Thisentryspawnsaloginshellontheconsole #Respawnmeansitwillberestartedeachtimeitiskilled con:2:respawn:/bin/sh This very simple[52] inittab script describes three individual runlevels. Each run-level is associated with a script, which must be created by the developer for the desired actions in each runlevel.Whenthisfileisreadbyinit,thefirstscripttobeexecutedis/etc/rc.sysinit.Thisisdenoted bythesysinittag.Theninitentersrunlevel2,andexecutesthescriptdefinedforrunlevel2.Fromthis example,thiswouldbe/etc/init.d/runlvl2.startup.Asyoumightguessfromthe:wait:taginListing66,initwaitsuntilthescriptcompletesbeforecontinuing.Whentherunlevel2scriptcompletes,init spawnsashellontheconsole(throughthe/bin/shsymboliclink),asshowninthelastlineofListing 6-6. The respawn keyword instructs init to restart the shell each time it detects that it has exited. Listing6-7showswhatitlookslikeduringboot. Listing6-7.ExampleStartupMessages ... VFS:Mountedroot(nfsfilesystem). Freeinginitmemory:304K INIT:version2.78booting Thisisrc.sysinit INIT:Enteringrunlevel:2 Thisisrunlvl2.startup # The startup scripts in this example do nothing except announce themselves for illustrative purposes. Of course, in an actual system, these scripts enable features and services that do useful work! Given the simple configuration in this example, you would enable the services and applications for your particular widget in the /etc/init.d/runlvl2.startup script and do the reversedisableyourapplications,services,anddevicesinyourshutdownand/orrebootscripts.Inthe next section, we look at some typical system configurations and the required entries in the startup scriptstoenabletheseconfigurations. 6.3.2.ExampleWebServerStartupScript Althoughsimple,thisexamplestartupscriptisdesignedtoillustratethemechanismandguideyou in designing your own system startup and shutdown behavior. This example is based on busybox, whichhasaslightlydifferentinitializationbehaviorthaninit.Thesedifferencesarecoveredindetail inChapter11. In a typical embedded appliance that contains a web server, we might want several servers available for maintenance and remote access. In this example, we enable servers for HTTP and Telnet access (via inetd). Listing 6-8 contains a simple rc.sysinit script for our hypothetical web serverappliance. Listing6-8.WebServerrc.sysinit #!/bin/sh echo"Thisisrc.sysinit" busyboxmount-tprocnone/proc #Loadthesystemloggers syslogd klogd #EnablelegacyPTYsupportfortelnetd busyboxmkdir/dev/pts busyboxmknod/dev/ptmxc52 busyboxmount-tdevptsdevpts/dev/pts Inthissimpleinitializationscript,wefirstenabletheprocfilesystem.Thedetailsofthisuseful subsystem are covered in Chapter 9. Next we enable the system loggers so that we can capture systeminformationduringoperation.Thisisespeciallyusefulwhenthingsgowrong.Thelastentries enablesupportfortheUNIXPTYsubsystem,whichisrequiredfortheimplementationoftheTelnet serverusedforthisexample. Listing 6-9 contains the commands in the runlevel 2 startup script. This script contains the commandstoenableanyserviceswewanttohaveoperationalforourappliance. Listing6-9.ExampleRunlevel2StartupScript #!/bin/sh echo"Thisisrunlvl2.startup" echo"StartingInternetSuperserver" inetd echo"Startingwebserver" webs& Noticehowsimplethisrunlevel2startupscriptactuallyis.Firstweenabletheso-calledInternet superserver inetd, which intercepts and spawns services for common TCP/IP requests. In our example, we enabled Telnet services through a configuration file called /etc/inetd.conf. Then we executethewebserver,herecalledwebs.That'sallthereistoit.Althoughminimal,thisisaworking configurationforTelnetandwebservices. To complete thisconfiguration, youmightsupplyashutdown script(referbackto Listing6-6), which, in this case, would terminate the web server and the Internet superserver before system shutdown.Inourexamplescenario,thatissufficientforacleanshutdown. 6.4.InitialRAMDisk The Linux kernel contains a mechanism to mount an early root file system to perform certain startup-relatedsysteminitializationandconfiguration.ThismechanismisknownastheinitialRAM disk, or simply initrd. Support for this functionality must be compiled into the kernel. This kernel configuration option is found under Block Devices, RAM disk support in the kernel configuration utility.Figure6-1showsanexampleoftheconfigurationforinitrd. Figure6-1.Linuxkernelconfigurationutility 6.4.1.InitialRAMDiskPurpose TheinitialRAMdiskisasmallself-containedrootfilesystemthatusuallycontainsdirectivesto load specific device drivers before the completion of the boot cycle. In Linux workstation distributions such as Red Hat and Fedora Core, an initial RAM disk is used to load the device drivers for the EXT3 file system before mounting the real root file system. An initrd is frequently usedtoloadadevicedriverthatisrequiredinordertoaccesstherealrootfilesystem. 6.4.2.Bootingwithinitrd To use the initrd functionality, the bootloader gets involved on most architectures to pass the initrd image to the kernel. A common scenario is that the bootloader loads a compressed kernel imageintomemoryandthenloadsaninitrdimageintoanothersectionofavailablememory.Indoing so, it becomes the bootloader's responsibility to pass the load address of the initrd image to the kernel before passing control to it. The exact mechanism differs depending on the architecture, bootloader,andplatformimplementation.However,thekernelmustknowwheretheinitrdimageis locatedsoitcanloadit. Somearchitecturesandplatformsconstructasinglecompositebinaryimage.Thisschemeisused whenthebootloaderdoesnothavespecificLinuxsupportforloadinginitrdimages.Inthiscase,the kernel and initrd image are simply concatenated together. You will find reference to this type of composite image in the kernel makefiles as bootpImage. Presently, this is used only for arm architecture. Sohowdoesthekernelknowwheretofindtheinitrdimage?Unlessthereissomespecialmagic inthebootloader,itisusuallysufficientsimplytopasstheinitrdimagestartaddressandsizetothe kernel via the kernel command line. Here is an example of a kernel command line for a popular ARM-basedreferenceboardcontainingtheTIOMAP5912processor. console=ttyS0,115200root=/dev/nfs\ nfsroot=192.168.1.9:/home/chris/sandbox/omap-target\ initrd=0x10800000,0x14af47 The previous kernel command line has been separated into several lines to fit in the space provided.Inactualpractice,itisasingleline,withtheindividualelementsseparatedbyspaces.This kernelcommandlinedefinesthefollowingkernelbehavior: •SpecifyasingleconsoleondevicettyS0at115kilobaud •MountarootfilesystemviaNFS,thenetworkfilesystem •FindtheNFSrootfilesystemonhost192.168.1.9(fromdirectory/home/chris/sandbox/omaptarget) • Load and mount an initial ramdisk from physical memory location 0x10800000, which has a sizeof0x14AF47(1,355,591bytes) Oneadditionalnoteregardingthisexample:Almostuniversally,theinitrdimageiscompressed. Thesizespecifiedonthekernelcommandlineisthesizeofthecompressedimage. 6.4.3.BootloaderSupportforinitrd Let's look at a simple example based on the popular U-Boot bootloader running on an ARM processor.ThisbootloaderhasbeendesignedwithLinuxkernelsupport.UsingU-Boot,itiseasyto include an initrd image with the kernel image. Listing 6-10 examines a typical boot sequence containinganinitialramdiskimage. Listing6-10.BootingKernelwithRamdiskSupport #tftpboot0x10000000kernel-uImage ... Loadaddress:0x10000000 Loading:############################done Bytestransferred=1069092(105024hex) #tftpboot0x10800000initrd-uboot ... Loadaddress:0x10800000 Loading:###########################################done Bytestransferred=282575(44fcfhex) #bootm0x100000000x10800040 Uncompressingkernel.................done. ... RAMDISKdriverinitialized:16RAMdisksof16384Ksize1024blocksize ... RAMDISK:Compressedimagefoundatblock0 VFS:Mountedroot(ext2filesystem). Greetings:thisislinuxrcfromInitialRAMDisk Mounting/procfilesystem BusyBoxv1.00(2005.03.14-16:37+0000)Built-inshell(ash) Enter'help'foralistofbuilt-incommands. #(<<<<Busyboxcommandprompt) Here in Listing 6-10, we get a glimpse of the U-Boot bootloader, which we examine in more detailinthenextchapter.ThetftpbootcommandcausesU-Boottodownloadthekernelimagefroma tftpserver.Thekernelimageisdownloadedandplacedintothebaseofthistargetsystem'smemory at the 256MB address (0x10000000 hex[53]). Then a second image, the initial ramdisk image, is downloaded from a tftp server into memory at a higher memory address (256MB + 8MB, in this example).Finally,weissuetheU-Bootbootmcommand,whichisthe"bootfrommemory"command. Thebootmcommandtakestwoarguments:theaddressoftheLinuxkernelimage,optionallyfollowed byanaddressrepresentingthelocationoftheinitialramdiskimage. Take special note of one feature of the U-Boot bootloader. It fully supports loading kernel and ramdisk images over an Ethernet connection. This is a very useful development configuration. You cangetakernelandramdiskimageontoyourboardinotherwaysaswell.Youcanflashtheminto yourFlashmemoryusingahardware-basedflashprogrammingtool,oryoucanuseaserialportand download the kernel and file system images via RS-232. However, because these images are typicallylarge(akernelcanbeaboutamegabyte,andaramdiskcanbetensofmegabytes),youwill save a significant amount of engineering time if you invest in this Ethernet-based tftp download method.Whateverbootloaderyouchoose,makesureitsupportsnetworkdownloadofdevelopment images. 6.4.4.initrdMagic:linuxrc When the kernel boots, it detects the presence of the initrd image, and copies the compressed binaryfilefromthespecifiedphysicallocationinRAMintoaproperkernelramdiskandmountsitas therootfilesystem.Themagicoftheinitrdcomesfromthecontentsofaspecialfilewithintheinitrd image.Whenthekernelmountstheinitialramdisk,itlooksforaspecificfilecalledlinuxrc.Ittreats this file as a script file and proceeds to execute the commands contained therein. This mechanism enablesthesystemdesignertospecifythebehaviorofinitrd.Listing6-11containsasamplelinuxrc file. Listing6-11.ExamplelinuxrcFile #!/bin/sh echo'Greetings:thisis'linuxrc'fromInitialRamdisk' echo'Mounting/procfilesystem' mount-tproc/proc/proc busyboxsh Inpractice,thisfilewouldcontaindirectivesrequiredbeforewemounttherealrootfilesystem. One example might be to load CompactFlash drivers to obtain a real root file system from a CompactFlashdevice.Forpurposesofthisexample,wesimplyspawnabusyboxshellandhaltthe bootprocessforexamination.Youcanseethe#commandpromptfromListing6-10resultingfrom this busybox shell. If one were to type the exit command here, the kernel would continue its boot processuntilcomplete. Afterthekernelcopiestheramdiskfromphysicalmemoryintoakernelramdisk,itreturnsthis physicalmemorybacktotheavailablememorypool.Youcanthinkofthisastransferringtheinitrd imagefromphysicalmemoryatthehard-codedaddressintothekernel'sownvirtualmemory(inthe formofakernelramdiskdevice). One last comment about Listing 6-11: The mount command in which the /proc file system is mountedseemsredundantinitsuseofthewordproc.Thiscommandwouldalsowork: mount-tprocnone/proc Notice that the device field of the mount command has been changed to none. The mount commandignoresthedevicefieldbecausenophysicaldeviceisassociatedwiththeprocfilesystem. The-tprocisenoughtoinstructmounttomountthe/procfilesystemonthe/procmountpoint.Iuse theformerinvocationasamentalreminderthatweareactuallymountingthekernelpseudodevice (the/procfilesystem)on/proc.Themountcommandignoresthisargument.Usethemethodthatyou prefer. 6.4.5.TheinitrdPlumbing AspartoftheLinuxbootprocess,thekernelmustlocateandmountarootfilesystem.Lateinthe bootprocess,thekerneldecideswhatandwheretomountinafunctioncalledprepare_namespace(). Ifinitrdsupportisenabledinthekernel,asillustratedinFigure6-1,andthekernelcommandlineis so configured, the kernel decompresses the compressed initrd image from physical memory and eventuallycopiesthecontentsofthisfileintoaramdiskdevice(/dev/ram).Atthispoint,wehavea proper file system on a kernel ramdisk. After the file system has been read into the ramdisk, the kernel effectively mounts this ramdisk device as its root file system. Finally, the kernel spawns a kernelthreadtoexecutethelinuxrcfileontheinitrdimage.[54] When the linuxrc script has completed execution, the kernel unmounts the initrd and proceeds with the final stages of system boot. If the real root device has a directory called /initrd, Linux mountstheinitrdfilesystemonthispath(inthiscontext,calledamountpoint).Ifthisdirectorydoes notexistinthefinalrootfilesystem,theinitrdimageissimplydiscarded. Ifthekernelcommandlinecontainsaroot=parameterspecifyingaramdisk(root=/dev/ram0,for example), the previously described initrd behavior changes in two important ways. First, the processing of the linuxrc executable is skipped. Second, no attempt is made to mount another file systemasroot.ThismeansthatyoucanhaveaLinuxsystemwithinitrdastheonlyrootfilesystem. This is useful for minimal system configurations in which the only root file system is the ramdisk. Placing/dev/ram0onthekernelcommandlineallowsthefullsysteminitializationtocompletewith theinitrdasthefinalrootfilesystem. 6.4.6.BuildinganinitrdImage Constructingasuitablerootfilesystemimageisoneofthemorechallengingaspectsofembedded systems.Creatingaproperinitrdimageisevenmorechallengingbecauseitneedstobesmalland specialized.Forthissection,weexamineinitrdrequirementsandfilesystemcontents. Listing 6-12 was produced by running the tree utility on our example initrd image from this chapter. Listing6-12.ContentsofExampleinitrd . |--bin ||--busybox ||--echo->busybox ||--mount->busybox |'--sh->busybox |--dev ||--console ||--ram0 |'--ttyS0 |--etc |--linuxrc '--proc 4directories,8files As you can see, it is very small indeed; it takes up a little more than 500KB in uncompressed form.Sinceitisbasedonbusybox,ithasmanycapabilities.Becausebusyboxisstaticallylinked,it hasnodependenciesonanysystemlibraries.YouwilllearnmoreaboutbusyboxinChapter11. 6.5.Usinginitramfs initramfsisarelativelynew(Linux2.6)mechanismforexecutingearlyuserspaceprograms.Itis conceptually similar to initrd, as described in the previous section. Its purpose is also similar: to enableloadingofdriversthatmightberequiredbeforemountingtherealrootfilesystem.However, itdiffersinsignificantwaysfromtheinitrdmechanism. The technical implementation details differ significantly between initrd and initramfs. For example,initramfsisloadedbeforethecalltodo_basic_setup(),[55]whichprovidesamechanismfor loadingfirmwarefordevicesbeforeitsdriverhasbeenloaded.Formoredetails,theLinuxkernel documentation for this subsystem is relatively up-to-date. See .../Documentation/filesystems/ramfsrootfs-initramfs.txt. Fromapracticalperspective,initramfsismucheasiertouse.initramfsisacpioarchive,whereas initrd is a gzipped file system image. This simple difference contributes to the easy of use of initramfs.ItisintegratedintotheLinuxkernelsourcetreeandisbuiltautomaticallywhenyoubuild the kernel image. Making changes to it is far easier than building and loading a new initrd image. Listing 6-13 shows the contents of the Linux kernel .../usr directory, where the initramfs image is built.ThecontentsofListing6-13areshownafterakernelhasbeenbuilt. Listing6-13.KernelinitramfsBuildDirectory $ls-l total56 -rw-rw-r--1chrischris834Mar2511:13built-in.o -rwxrwxr-x1chrischris11512Mar2511:13gen_init_cpio -rw-rw-r--1chrischris10587Oct272005gen_init_cpio.c -rw-rw-r--1chrischris512Mar2511:13initramfs_data.cpio -rw-rw-r--1chrischris133Mar2511:13initramfs_data.cpio.gz -rw-rw-r--1chrischris786Mar2511:13initramfs_data.o -rw-rw-r--1chrischris1024Oct272005initramfs_data.S -rw-rw-r--1chrischris113Mar2511:13initramfs_list -rw-rw-r--1chrischris1619Oct272005Kconfig -rw-rw-r--1chrischris2048Oct272005Makefile Thefileinitramfs_listcontainsalistoffilesthatwillbeincludedintheinitramfsarchive.The defaultforrecentLinuxkernelslookslikethis: dir/dev075500 nod/dev/console060000c51 dir/root070000 This produces a small default directory structure containing the /root and /dev top-level directories,aswellasasingledevicenoderepresentingtheconsole.Addtothisfiletobuildyour own initramfs. You can also specify a source for your initramfs files via the kernel-configuration facility.EnableINITRAMFS_SOURCEinyourkernelconfigurationandpointittoalocationonyour developmentworkstation;thekernelbuildsystemwillusethosefilesasthesourceforyourinitramfs image. The finaloutputofthis build directoryistheinitramfs_data_cpio.gzfile.Thisis acompressed archive containing the files you specified (either through the initramfs_list or via the INITRAMFS_SOURCE kernel-configuration option). This archive is linked into the final kernel image.Thisisanotheradvantageofinitramfsoverinitrd:Thereisnoneedtoloadaseparateinitrd imageatboottime,asisthecasewithinitrd. 6.6.Shutdown Orderly shutdown of an embedded system is often overlooked in a design. Improper shutdown can affect startup times and can even corrupt certain file system types. One of the more common complaintsusingtheEXT2filesystem(thedefaultinmanydesktopLinuxdistributionsforseveral years) is the time it takes for an fsck (file system check) on startup after unplanned power loss. Serverswithlargedisksystemscantakeontheorderofhourstoproperlyfsckthroughacollectionof largeEXT2partitions. Eachembeddedprojectwilllikelyhaveitsownshutdownstrategy.Whatworksforonemightor mightnotworkforanother.ThescaleofshutdowncanrangefromafullSystemVshutdownscheme, to a simple script to halt or reboot. Several Linux utilities are available to assist in the shutdown process,includingtheshutdown,halt,andrebootcommands.Ofcourse,thesemustbeavailablefor yourchosenarchitecture. A shutdown script should terminate all userspace processes, which results in closing any open files used by those processes. If init is being used, issuing the command init 0 halts the system. In general,theshutdownprocessfirstsendsallprocessestheSIGTERMsignal,tonotifythemthatthe system is shutting down. A short delay ensures that all processes have the opportunity to perform theirshutdownactions,suchasclosingfiles,savingstate,andsoon.Thenallprocessesaresentthe SIGKILLsignal,whichresultsintheirtermination.Theshutdownprocessshouldattempttounmount any mounted file systems and call the architecture-specific halt or reboot routines. The Linux shutdowncommandinconjunctionwithinitexhibitsthisbehavior. 6.7.ChapterSummary •ArootfilesystemisrequiredforallLinuxsystems.Theycanbedifficulttobuildfromscratch becauseofcomplexdependenciesbyeachapplication. • The File System Hierarchy standard provides guidance to developers for laying out a file systemformaximumcompatibilityandflexibility. •Wepresentedaminimalfilesystemasanexampleofhowrootfilesystemsarecreated. •TheLinuxkernel'sfinalbootstepsdefineandcontrolaLinuxsystem'sstartupbehavior.Several mechanismsareavailabledependingonyourembeddedLinuxsystem'srequirements. •Theinitprocesswaspresentedindetail.Thispowerfulsystem-configurationandcontrolutility canserveasthebasisforyourownembeddedLinuxsystem.Systeminitializationbasedoninitwas presented,alongwithexamplestartupscriptconfigurations. •InitialramdiskisaLinuxkernelfeaturetoallowfurtherstartupbehaviorcustomizationbefore mounting a final root file system and spawning init. We presented the mechanism and example configurationforusingthispowerfulfeature. • initramfs simplifies the initial ramdisk mechanism, while providing similar early startup facilities. It is easier to use, does not require loading a separate image, and is built automatically duringeachkernelbuild. 6.7.1.SuggestionsforAdditionalReading FileSystemHierarchyStandard Maintainedbyfreestandards.org www.pathname.com/fhs/ BootProcess,InitandShutdown LinuxDocumentationProject http://tldp.org/LDP/intro-linux/html/sect_04_02.html Initmanpage LinuxDocumentationProject http://tldp.org/LDP/sag/html/init.html AbriefdescriptionofSystemVinit http://docs.kde.org/en/3.3/kdeadmin/ksysv/what-is-sysv-init.html BootingLinux:TheHistoryandtheFuture WernerAlmesberger www.almesberger.net/cv/papers/ols2k-9.ps Chapter7.Bootloaders Previouschaptershavemadereferencetoandevenprovidedexamplesofbootloaderoperations. Acriticalcomponentofanembeddedsystem,thebootloaderprovidesthefoundationfromwhichthe othersystemsoftwareisspawned.Thischapterstartsbyexaminingthebootloader'sroleinasystem. We follow this with an introduction to some common features of bootloaders. Armed with this background, we take a detailed look at a popular bootloader used for embedded systems. We concludethischapterbyintroducingafewofthemorepopularbootloaders. Numerousbootloadersareinusetoday.Itwouldbeimpracticalinthegivenspacetocovermuch detail on even the most popular ones. Therefore, we have chosen to explain concepts and use examplesbasedononeofthemorepopularbootloadersintheopensourcecommunityforPowerPC, MIPS,ARM,andotherarchitectures:theU-Bootbootloader. 7.1.RoleofaBootloader Whenpowerisfirstappliedtoaprocessorboard,manyelementsofhardwaremustbeinitialized beforeeventhesimplestprogramcanrun.Eacharchitectureandprocessorhasasetofpredefined actionsandconfigurations,whichincludefetchingsomeinitializationcodefromanon-boardstorage device (usually Flash memory). This early initialization code is part of the bootloader and is responsibleforbreathinglifeintotheprocessorandrelatedhardwarecomponents. Most processors have a default address from which the first bytes of code are fetched upon application of power and release of reset. Hardware designers use this information to arrange the layout of Flash memory on the board and to select which address range(s) the Flash memory responds to. This way, when power is first applied, code is fetched from a well-known and predictableaddress,andsoftwarecontrolcanbeestablished. Thebootloaderprovidesthisearlyinitializationcodeandisresponsibleforinitializingtheboard so that other programs can run. This early initialization code is almost always written in the processor'snativeassemblylanguage.Thisfactalonepresentsmanychallenges,someofwhichwe examinehere. Ofcourse,afterthebootloaderhasperformedthisbasicprocessorandplatforminitialization,its primaryrolebecomesbootingafull-blownoperatingsystem.Itisresponsibleforlocating,loading, and passing execution to the primary operating system. In addition, the bootloader might have advancedfeatures,suchasthecapabilitytovalidateanOSimage,thecapabilitytoupgradeitselfor an OS image, and the capability to choose from among several OS images based on a developerdefinedpolicy.UnlikethetraditionalPC-BIOSmodel,whentheOStakescontrol,thebootloaderis overwrittenandceasestoexist.[56] 7.2.BootloaderChallenges Evenasimple"HelloWorld"programwritteninCrequiressignificanthardwareandsoftware resources. The application developer does not need to know or care much about these details becausetheCruntimeenvironmenttransparentlyprovidesthisinfrastructure.Abootloaderdeveloper has no such luxury. Every resource that a bootloader requires must be carefully initialized and allocated before it is used. One of the most visible examples of this is Dynamic Random Access Memory(DRAM). 7.2.1.DRAMController DRAMchipscannotbedirectlyreadfromorwrittentolikeothermicroprocessorbusresources. Theyrequirespecializedhardwarecontrollerstoenablereadandwritecycles.Tofurthercomplicate matters, DRAM must be constantly refreshed or the data contained within will be lost. Refresh is accomplishedbysequentiallyreadingeachlocationinDRAMinasystematicmannerandwithinthe timing specifications set forth by the DRAM manufacturer. Modern DRAM chips support many modesofoperation,suchasburstmodeanddualdatarateforhigh-performanceapplications.Itisthe DRAM controller's responsibility to configure DRAM, keep it refreshed within the manufacturer's timingspecifications,andrespondtothevariousreadandwritecommandsfromtheprocessor. SettingupaDRAMcontrolleristhesourceofmuchfrustrationforthenewcomertoembedded development.ItrequiresdetailedknowledgeofDRAMarchitecture,thecontrolleritself,thespecific DRAMchipsbeingused,andtheoverallhardwaredesign.Thoughthisisbeyondthescopeofthis book,theinterestedreadercanlearnmoreaboutthisimportantconceptbyreferringtothereferences at the end of this chapter. Appendix D, "SDRAM Interface Considerations," provides more backgroundonthisimportanttopic. VerylittlecanhappeninanembeddedsystemuntiltheDRAMcontrollerandDRAMitselfhave been properly initialized. One of the first things a bootloader must do is to enable the memory subsystem.Afteritisinitialized,memorycanbeusedasaresource.Infact,oneofthefirstactions many bootloaders perform after memory initialization is to copy themselves into DRAM for faster execution. 7.2.2.FlashVersusRAM Anothercomplexityinherentinbootloadersisthattheyarerequiredtobestoredinnonvolatile storagebutareusuallyloadedintoRAMforexecution.Again,thecomplexityarisesfromthelevelof resourcesavailableforthebootloadertorelyon.Inafullyoperationalcomputersystemrunningan operating system such as Linux, it is relatively easy to compile a program and invoke it from nonvolatilestorage.Theruntimelibraries,operatingsystem,andcompilerworktogethertocreatethe infrastructurenecessarytoloadaprogramfromnonvolatilestorageintomemoryandpasscontrolto it. The aforementioned "Hello World" program is a perfect example. When compiled, it can be loaded into memory and executed simply by typing the name of the executable (hello) on the commandline(assuming,ofcourse,thattheexecutableexistssomewhereonyourPATH). This infrastructure does not exist when a bootloader gains control upon power-on. Instead, the bootloadermustcreateitsownoperationalcontextandmoveitself,ifrequired,toasuitablelocation inRAM.Furthermore,additionalcomplexityisintroducedbytherequirementtoexecutefromareadonlymedium. 7.2.3.ImageComplexity As application developers, we do not need to concern ourselves with the layout of a binary executable file when we develop applications for our favorite platform. The compiler and binary utilities are preconfigured to build a binary executable image containing the proper components neededforagivenarchitecture.Thelinkerplacesstartup(prologue)andshutdown(epilogue)code into the image. These objects set up the proper execution context for your application, which typicallystartsatmain()inyourapplication. Thisisabsolutelynotthecasewithatypicalbootloader.Whenthebootloadergetscontrol,there isnocontextorpriorexecutionenvironment.Inatypicalsystem,theremightnotbeanyDRAMuntil thebootloaderinitializestheprocessorandrelatedhardware.Considerwhatthismeans.Inatypical Cfunction,anylocalvariablesarestoredonthestack,soasimplefunctionliketheoneinListing7-1 isunusable. Listing7-1.SimpleCfunction intsetup_memory_controller(board_info_t*p){ unsignedint*dram_controller_register=p->dc_reg; ... Whenabootloadergainscontrolonpower-on,thereisnostackandnostackpointer.Therefore,a simple C function similar to Listing 7-1 will likely crash the processor because the compiler will generatecodetocreateandinitializethepointerdram_controller_registeronthestack,whichdoes notyetexist.ThebootloadermustcreatethisexecutioncontextbeforeanyCfunctionsarecalled. Whenthebootloaderiscompiledandlinked,thedevelopermustexercisecompletecontrolover howtheimageisconstructedandlinked.Thisisespeciallytrueifthebootloaderistorelocateitself from Flash to RAM. The compiler and linker must be passed a handful of parameters defining the characteristicsandlayoutofthefinalexecutableimage.Twoprimarycharacteristicsconspiretoadd complexitytothefinalbinaryexecutableimage. The first characteristic that presents complexity is the need to organize the startup code in a formatcompatiblewiththeprocessor'sbootsequence.Thefirstbytesofexecutablecodemustbeata predefined location in Flash, depending on the processor and hardware architecture. For example, theAMCCPowerPC405GPprocessorseeksitsfirstmachineinstructionsfromahard-codedaddress of0xFFFF_FFFC.Otherprocessorsusesimilarmethodswithdifferentdetails.Someprocessorsare configurable at power-on to seek code from one of several predefined locations, depending on hardwareconfigurationsignals. How does a developer specify the layout of a binary image? The linker is passed a linker descriptionfile,alsocalledalinkercommandscript.Thisspecialfilecanbethoughtofasarecipe for constructing a binary executable image. Listing 7-2 contains a snippet from an existing linker descriptionfileinuseinapopularbootloader,whichwediscussshortly. Listing7-2.LinkerCommandScriptResetVectorPlacement SECTIONS { .resetvec0xFFFFFFFC: { *(.resetvec) }=0xffff ... Acompletedescriptionoflinkercommandscriptssyntaxisbeyondthescopeofthisbook.The interestedreaderisdirectedtotheGNULDmanualreferencedattheendofthischapter.Lookingat Listing7-2,weseethebeginningofthedefinitionfortheoutputsectionofthebinaryELFimage.It directsthelinkertoplacethesectionofcodecalled.resetvecatafixedaddressintheoutputimage, startingatlocation0xFFFF_FFFC.Furthermore,itspecifiesthattherestofthissectionshallbefilled with all ones (0xFFFF.) This is because an erased Flash memory array contains all ones. This technique not only saves wear and tear on the Flash memory, but it also significantly speeds up programmingofthatsector. Listing7-3isthecompleteassemblylanguagefilefromarecentU-Bootdistributionthatdefines the .resetvec code section. It is contained in an assembly language file called .../cpu/ppc4xx/resetvec.S.Noticethatthiscodesectioncannotexceed4bytesinlengthinamachine with only 32 address bits. This is because only a single instruction is defined in this section, no matterwhatconfigurationoptionsarepresent. Listing7-3.SourceDefinitionof.resetvec /*CopyrightMontaVistaSoftwareIncorporated,2000*/ #include<config.h> .section.resetvec,"ax" #ifdefined(CONFIG_440) b_start_440 #else #ifdefined(CONFIG_BOOT_PCI)&&defined(CONFIG_MIP405) b_start_pci #else b_start #endif #endif Thisassemblylanguagefileisveryeasytounderstand,evenifyouhavenoassemblylanguage programmingexperience.Dependingontheparticularconfiguration(asspecifiedbytheCONFIG_* macros), an unconditional branch instruction (b in PowerPC assembler syntax) is generated to the appropriate start location in the main body of code. This branch location is a 4-byte PowerPC instruction,andaswesawinthesnippetfromthelinkercommandscriptinListing7-2,thissimple branchinstructionisplacedintheabsoluteFlashaddressof0xFFFF_FFFCintheoutputimage.As mentioned earlier, the PPC 405GP processor fetches its first instruction from this hard-coded address. This is how the first sequence of code is defined and provided by the developer for this particulararchitectureandprocessorcombination. 7.2.4.ExecutionContext Theotherprimaryreasonforbootloaderimagecomplexityisthelackofexecutioncontext.When thesequenceofinstructionsfromListing7-3startsexecuting(recallthatthesearethefirstmachine instructionsafterpower-on),theresourcesavailabletotherunningprogramarenearlyzero.Default valuesdesignedintothehardwareensurethatfetchesfromFlashmemoryworkproperlyandthatthe system clock has some default values, but little else can be assumed.[57] The reset state of each processorisusuallywelldefinedbythemanufacturer,buttheresetstateofaboardisdefinedbythe hardwaredesigners. Indeed,mostprocessorshavenoDRAMavailableatstartupfortemporarystorageofvariables or,worse,forastackthatisrequiredtouseCprogramcallingconventions.Ifyouwereforcedto writea"HelloWorld"programwithnoDRAMand,therefore,nostack,itwouldbequitedifferent fromthetraditional"HelloWorld"example. Thislimitationplacessignificantchallengesontheinitialbodyofcodedesignedtoinitializethe hardware.Asaresult,oneofthefirsttasksthebootloaderperformsonstartupistoconfigureenough of the hardware to enable at least some minimal amount of RAM. Some processors designed for embeddedusehavesmallamountsofon-chipstaticRAMavailable.ThisisthecasewiththePPC 405GPwe'vebeendiscussing.WhenRAMisavailable,astackcanbeallocatedusingpartofthat RAM,andapropercontextcanbeconstructedtorunhigher-levellanguagessuchasC.Thisallows the rest of the processor and platform initialization to be written in something other than assembly language. 7.3.AUniversalBootloader:DasU-Boot Many open-source and commercial bootloaders are available, and many more one-of-a-kind home-growndesignsareinwidespreadusetoday.Mostofthesehavesomelevelofcommonalityof features. For example, all of them have some capability to load and execute other programs, particularlyanoperatingsystem.Mostinteractwiththeuserthroughaserialport.Supportforvarious networkingsubsystems(suchasEthernet)islesscommonbutaverypowerfulfeature. Many bootloaders are specific to a particular architecture. The capability of a bootloader to support a wide variety of architectures and processors can be an important feature to larger development organizations. It is not uncommon for a single development organization to have multiple processors spanning more than one architecture. Investing in a single bootloader across multipleplatformsultimatelyresultsinlowerdevelopmentcosts. In this section, we study an existing bootloader that has become very popular in the embedded Linuxcommunity.TheofficialnameforthisbootloaderisDasU-Boot.ItismaintainedbyWolfgang Denk and hosted on SourceForge at http://u-boot.sourceforge.net/. U-Boot has support for multiple architectures and has a large following of embedded developers and hardware manufacturers who haveadopteditforuseintheirprojectsandhavecontributedtoitsdevelopment. 7.3.1.SystemConfiguration:U-Boot For a bootloader to be useful across many processors and architectures, some method of configuringthebootloaderisnecessary.AswiththeLinuxkernelitself,configurationofabootloader isdoneatcompiletime.Thismethodsignificantlyreducesthecomplexityofthebootloader,which, initself,isanimportantcharacteristic. InthecaseofU-Boot,board-specificconfigurationisdrivenbyasingleheaderfilespecificto thetargetplatform,andafewsoftlinksinthesourcetreethatselectthecorrectsubdirectoriesbased ontargetboard,architecture,andCPU.WhenconfiguringU-Bootforoneofitssupportedplatforms, issuethiscommand: $make<platform>_config Here,platformisoneofthemanyplatformssupportedbyU-Boot.Theseplatform-configuration targetsarelistedinthetoplevelU-Bootmakefile.Forexample,toconfigurefortheSpectrumDigital OSK,whichcontainsaTIOMAP5912processor,issuethiscommand: $makeomap5912osk_config ThisconfigurestheU-BootsourcetreewiththeappropriatesoftlinkstoselectARMasthetarget architecture,theARM926core,andthe5912OSKasthetargetplatform. ThenextstepinconfiguringU-Bootforthisplatformistoedittheconfigurationfilespecificto this board. This file is found in the U-Boot ../include/configs subdirectory and is called omap5912osk.h.TheREADMEfilethatcomeswiththeU-Bootdistributiondescribesthedetailsof configurationandisthebestsourceforthisinformation. ConfigurationofU-Bootisdoneusingconfigurationvariablesdefinedinaboard-specificheader file.Configurationvariableshavetwoforms.Configurationoptionsareselectedusingmacrosinthe form of CONFIG_XXXX. Configuration settings are selected using macros in the form of CFG_XXXX. In general, configuration options (CONFIG_XXX) are user configurable and enable specific U-Boot operational features. Configuration settings (CFG_XXX) are usually hardware specific and require detailed knowledge of the underlying processor and/or hardware platform. Board-specific U-Boot configuration is driven by a header file dedicated to that specific platform thatcontainsconfigurationoptionsandsettingsappropriatefortheunderlyingplatform.TheU-Boot sourcetreeincludesadirectorywheretheseboard-specificconfigurationheaderfilesreside.They canbefoundin.../include/configsfromthetop-levelU-Bootsourcedirectory. Numerous features and modes of operation can be selected by adding definitions to the boardconfigurationfile.Listing7-4containsapartialconfigurationheaderfileforafictitiousboardbased onthePPC405GPprocessor. Listing7-4.PartialU-BootBoard-ConfigurationHeaderFile #defineCONFIG_405GP/*Processordefinition*/ #defineCONFIG_4XX/*Sub-archspecification,4xxfamily*/ #defineCONFIG_SYS_CLK_FREQ33333333/*PLLFrequency*/ #defineCONFIG_BAUDRATE9600 #defineCONFIG_PCI/*EnablesupportforPCI*/ ... #defineCONFIG_COMMANDS(CONFIG_CMD_DFL|CFG_CMD_DHCP) ... #defineCFG_BASE_BAUD691200 /*Thefollowingtableincludesthesupportedbaudrates*/ #defineCFG_BAUDRATE_TABLE\ {1200,2400,4800,9600,19200,38400,57600,115200,230400} #defineCFG_LOAD_ADDR0x100000/*defaultloadaddress*/ ... /*MemoryBank0(FlashBank0)initialization*/ #defineCFG_EBC_PB0AP0x9B015480 #defineCFG_EBC_PB0CR0xFFF18000 #defineCFG_EBC_PB1AP0x02815480 #defineCFG_EBC_PB1CR0xF0018000 ... Listing7-4givesanideaofhowU-Bootitselfisconfiguredforagivenboard.Anactualboardconfigurationfilecancontainhundredsoflinessimilartothosefoundhere.Inthisexample,youcan seethedefinitionsfortheCPU,CPUfamily(4xx),PLLclockfrequency,serialportbaudrate,and PCI support. We have included examples of configuration variables (CONFIG_XXX) and configurationsettings(CFG_XXX).Thelastfewlinesareactualprocessorregistervaluesrequired toinitializetheexternalbuscontrollerformemorybanks0and1.Youcanseethatthesevaluescan comeonlyfromadetailedknowledgeoftheboardandprocessor. ManyaspectsofU-Bootcanbeconfiguredusingthesemechanisms,includingwhatfunctionality willbecompiledintoU-Boot(supportforDHCP,memorytests,debuggingsupport,andsoon).This mechanismcanbeusedtotellU-Boothowmuchandwhatkindofmemoryisonagivenboard,and wherethatmemoryismapped.TheinterestedreadercanlearnmuchmorebylookingattheU-Boot codedirectly,especiallytheexcellentREADMEfile. 7.3.2.U-BootCommandSets U-Boot supports more than 60 standard command sets that enable more than 150 unique commands using CFG_* macros. A command set is enabled in U-Boot through the use of configuration setting (CFG_*) macros. For a complete list from a recent U-Boot snapshot, consult Appendix B, "U-Boot Configurable Commands." Here are just a few, to give you an idea of the capabilitiesavailable: CommandSet Commands CFG_CMD_FLASH Flashmemorycommands CFG_CMD_MEMORY Memorydump,fill,copy,compare,andsoon CFG_CMD_DHCP DHCPSupport CFG_CMD_PING Pingsupport CFG_CMD_EXT2 EXT2Filesystemsupport ThefollowinglineofListing7-4definesthecommandsenabledinagivenU-Bootconfiguration, asdrivenbytheboard-specificheaderfile: #defineCONFIG_COMMANDS(CONFIG_CMD_DFL|CFG_CMD_DHCP) Instead of typing out each individual CFG_* macro in your own board-specific configuration header, you can start from a default set of commands predefined in the U-Boot source. The macro CONFIG_CMD_DFLdefinesthisdefaultsetofcommands.CONFIG_CMD_DFLspecifiesalistof default U-Boot command sets such as tftpboot (boot an image from a tftpserver), bootm (boot an image from memory), memory utilities such as md (display memory), and so on. To enable your specificcombinationofcommands,youcanstartwiththedefaultandaddandsubtractasnecessary. The example from Listing 7-4 adds the DHCP command set to the default. You can subtract in a similarfashion: #defineCONFIG_COMMANDS(CONFIG_CMD_DFL&~CFG_CMD_NFS) Takealookatanyboard-configurationheaderfilein.../include/configs/forexamples. 7.3.3.NetworkOperations ManybootloadersincludesupportforEthernetinterfaces.Inadevelopmentenvironment,thisisa hugetimesaver.Loadingevenamodestkernelimageoveraserialportcantakeminutesversusafew secondsovera10MbpsEthernetlink.Furthermore,seriallinksaremorepronetoerrorsfrompoorly behavedserialterminals. SomeofthemoreimportantfeaturestolookforinabootloaderincludesupportfortheBOOTP, DHCP, and TFTP protocols. For those unfamiliar with these, BOOTP (Bootstrap Protocol) and DHCP(DynamicHostControlProtocol)areprotocolsthatenableatargetdevicewithanEthernet port to obtain an IP address and other network-related configuration information from a central server.TFTP(TrivialFileTransferProtocol)allowsthetargetdevicetodownloadfiles(suchasa Linuxkernelimage)fromaTFTPserver.Referencestotheseprotocolspecificationsarelistedatthe endofthischapter.ServersfortheseservicesaredescribedinChapter12,"EmbeddedDevelopment Environment." Figure7-1illustratestheflowofinformationbetweenthetargetdeviceandaBOOTPserver.The client(U-Boot,inthiscase)initiatesabroadcastpacketsearchingforaBOOTPserver.Theserver responds with a reply packet that includes the client's IP address and other information. The most usefuldataincludesafilenameusedtodownloadakernelimage. Figure7-1.BOOTPclient/serverhandshake In practice, dedicated BOOTP servers no longer exist as stand-alone servers. DHCP servers includedwithyourfavoriteLinuxdistributionalsosupportBOOTPprotocolpackets. The DHCP protocol builds upon BOOTP. It can supply the target with a wide variety of configuration information. In practice, the information exchange is often limited by the target/bootloader DHCP client implementation. Listing 7-5 contains an example of a DHCP server configurationblockidentifyingasingletargetdevice.ThisisasnippetfromaDHCPconfiguration filefromtheFedoraCore2DHCPimplementation. Listing7-5.DHCPTargetSpecification hostcoyote{ hardwareethernet00:0e:0c:00:82:f8; netmask255.255.255.0; fixed-address192.168.1.21; server-name192.168.1.9; filename"coyote-zImage"; optionroot-path"/home/chris/sandbox/coyote-target"; } ... WhenthisDHCPserverreceivesapacketfromadevicematchingthehardwareEthernetaddress containedinListing7-5,itrespondsbysendingthatdevicetheparametersinthistargetspecification. Table7-1describesthefieldsinthetargetspecification. Table7-1.DHCPTargetParameters DHCPTarget Purpose Comments Parameter host Hostname SymboliclabelfromDHCPconfigurationfile Ethernet hardware Low-levelEthernethardwareaddressofthetarget'sEthernet hardware ethernet interface address fixed-address TargetIPaddress TheIPaddressthatthetargetwillassume netmask Targetnetmask TheIPnetmaskthatthetargetwillassume TFTPserverIP TheIPaddresstowhichthetargetwilldirectrequestsforfile server-name address transfers,rootfilesystem,andsoon Thefilenamethatthebootloadercanusetobootasecondary filename TFTPfilename image(usuallyaLinuxkernel) When the bootloader on the target board has completed the BOOTP or DHCP exchange, the parametersdescribedpreviouslyareusedforfurtherconfiguration.Forexample,thebootloaderuses the target IP address to bind its Ethernet port with this IP address. The bootloader then uses the server-name field as a destination IP address to request the file contained in the filename field, which, in most cases, represents a Linux kernel image. Although this is the most common use, this samescenariocouldbeusedtodownloadandexecutemanufacturingtestanddiagnosticsfirmware. ItshouldbenotedthattheDHCPprotocolsupportsmanymoreparametersthanthosedetailedin Table 7-1. These are simply the more common parameters you might encounter for embedded systems.SeetheDHCPspecificationreferencedattheendofthischapterforcompletedetails. 7.3.4.StorageSubsystems Manybootloaderssupportthecapabilityofbootingimagesfromavarietyofnonvolatilestorage devicesinadditiontotheusualFlashmemory.Thedifficultyinsupportingthesetypesofdevicesis therelativecomplexityinbothhardwareandsoftware.Toaccessdataonaharddrive,forexample, thebootloadermusthavedevicedrivercodefortheIDEcontrollerinterface,aswellasknowledge of the underlying partition scheme and file system. This is not trivial and is one of the tasks more suitedtofull-blownoperatingsystems. Evenwiththeunderlyingcomplexity,methodsexistforloadingimagesfromthisclassofdevice. Thesimplestmethodistosupportthehardwareonly.Inthisscheme,noknowledgeofthefilesystem isassumed.Thebootloadersimplyraw-loadsfromabsolutesectorsonthedevice.Thisschemecan beusedbydedicatinganunformattedpartitionfromsector0onanIDE-compatibledevice(suchas CompactFlash)andloadingthedatafoundtherewithoutanystructureimposedonthedata.Thisisan ideal configuration for loading a kernel image or other binary image. Additional partitions on the device can be formatted for a given file system and can contain complete file systems. After the kernelboots,devicedriverscanbeusedtoaccesstheadditionalpartitions. U-Bootcanloadanimagefromaspecifiedrawpartitionorfromapartitionwithafilesystem structure.Ofcourse,theboardmusthaveasupportedhardwaredevice(anIDEsubsystem)andUBootmustbesoconfigured.AddingCFG_CMD_IDEtotheboard-specificconfigurationfileenables support for an IDE interface, and adding CFG_CMD_BOOTD enables support for booting from a raw partition. If you are porting U-Boot to a custom board, you will have to modify U-Boot to understandyourparticularhardware. 7.3.5.BootingfromDisk:U-Boot As described in the previous section, U-Boot supports several methods for booting a kernel imagefromadisksubsystem.Thissimplecommandillustratesoneofthesupportedmethods: =>diskboot0x4000000:0 Tounderstandthissyntax,youmustfirstunderstandhowU-Bootnumbersdiskdevices.The0:0 in this example specifies the device and partition. In this simple example, U-Boot performs a raw binaryloadoftheimagefoundonthefirstIDEdevice(IDEdevice0)fromthefirstpartitionfoundon thisdevice.Theimageisloadedintosystemmemoryatphysicaladdress0x400000. After the kernel image has been loaded into memory, the U-Boot bootm command (boot from memory)isusedtobootthekernel: =>bootm0x400000 7.4.PortingU-Boot One of the reasons U-Boot has become so popular is the ease in which new platforms can be supported. Each board port must supply a subordinate makefile that supplies board-specific definitions to the build process. These files are all given the name config.mk and exist in the .../board/xxx subdirectory under the U-Boot top-level source directory, where xxx specifies a particularboard. As of a recent U-Boot 1.1.4 snapshot, more than 240 different board configuration files are named config.mk under the .../boards subdirectory. In this same U-Boot version, 29 different CPU configurations are supported (counted in the same manner). Note that, in some cases, the CPU configurationcoversafamilyofchips,suchasppc4xx,whichhassupportforseveralprocessorsin thePowerPC4xxfamily.U-BootsupportsalargevarietyofpopularCPUsandCPUfamiliesinuse today,andamuchlargercollectionofreferenceboardsbasedontheseprocessors. IfyourboardcontainsoneofthesupportedCPUs,portingU-Bootisquitestraightforward.Ifyou mustaddanewCPU,planonsignificantlymoreeffort.Thegoodnewsisthatsomeonebeforeyou hasprobablydonethebulkofthework.WhetheryouareportingtoanewCPUoranewboardbased on an existing CPU, study the existing source code for specific guidance. Determine what CPU is closesttoyours,andclonethefunctionalityfoundinthatCPU-specificdirectory.Finally,modifythe resultingsourcestoaddthespecificsupportforyournewCPU'srequirements. 7.4.1.EP405U-BootPort ThesamelogicappliestoportingU-Boottoanewboard.Let'slookatanexample.Wewilluse the Embedded Planet EP405 board, which contains the AMCC PowerPC 405GP processor. The particular board used for this example was provided courtesy of Embedded Planet and came with 64MBofSDRAMand16MBofon-boardFlash.Numerousotherdevicescompletethedesign. Thefirststepistoseehowclosewecancometoanexistingboard.ManyboardsintheU-Boot source tree support the 405GP processor. A quick grep of the board-configuration header files narrowsthechoicestothosethatsupportthe405GPprocessor: $cd.../u-boot/include/configs$grep-lCONFIG_405GP* In a recent U-Boot snapshot, 25 board configuration files are configured for 405GP. After examining a few, the AR405.h configuration is chosen as a baseline. It contains support for the LXT971Ethernettransceiver,whichisalsoontheEP405.Thegoalistominimizeanydevelopment workbyborrowingfromothersinthespiritofopensource.Let'stackletheeasystepsfirst.Copythe board-configuration file to a new file with a name appropriate for your board. We'll call ours EP405.h.Thesecommandsareissuedfromthetop-levelU-Bootsourcetree. $cp.../include/configs/AR405.h.../include/configs/EP405.h Then create the board-specific directory and make a copy of the AR405 board files. We don't know yet whether we need all of them. That step comes later. After copying the files to your new boarddirectory,editthefilenamesappropriatelyforyourboardname. $cdboard<<<fromtoplevelU-Bootsourcedirectory $mkdirep405 $cpesd/ar405/*ep405 Now comes the hard part. Jerry Van Baren, a developer and U-Boot contributor, detailed a humorousthoughrealisticprocessforportingU-Bootinane-mailpostingtotheU-Bootmailinglist. Hiscompleteprocess,documentedinC,canbefoundintheU-BootREADMEfile.Thefollowing summarizesthehardpartoftheportingprocessinJerry'sstyleandspirit: while(!running){ do{ Add/modifysourcecode }until(compiles); Debug; ... } Jerry'sprocess,assummarizedhere,isthesimpletruth.Whenyouhaveselectedabaselinefrom whichtoport,youmustadd,delete,andmodifysourcecodeuntilitcompiles,andthendebugituntil it is running without error! There is no magic formula. Porting any bootloader to a new board requires knowledge of many areas of hardware and software. Some of these disciplines, such as setting up SDRAM controllers, are rather specialized and complex. Virtually all of this work involves a detailed knowledge of the underlying hardware. The net result: Be prepared to spend manyentertaininghoursporingoveryourprocessor'shardwarereferencemanual,alongwiththedata sheetsofnumerousothercomponentsthatresideonyourboard. 7.4.2.U-BootMakefileConfigurationTarget Nowthatwehaveacodebasetostartfrom,wemustmakesomemodificationstothetop-level U-Bootmakefiletoaddtheconfigurationstepsforournewboard.Uponexaminingthismakefile,we findasectionforconfiguringtheU-Bootsourcetreeforthevarioussupportedboards.Wenowadd supportforournewonesowecanbuildit.BecausewederivedourboardfromtheESDAR405,we willusethatruleasthetemplateforbuildingourown.IfyoufollowalongintheU-Bootsourcecode, youwillseethattheserulesareplacedinthemakefileinalphabeticalorderoftheirconfiguration name.Weshallbegoodopen-sourcecitizensandfollowthatlead.Wecallourconfigurationtarget EP405_config,againinconcertwiththeU-Bootconventions. EBONY_config:unconfig @./mkconfig$(@:_config=)ppcppc4xxebony +EP405_config:unconfig +@./mkconfig$(@:_config=)ppcppc4xxep405 + ERIC_config:unconfig @./mkconfig$(@:_config=)ppcppc4xxeric Our new configuration rule has been inserted as shown in the three lines preceded with the + character(unifieddiffformat). Uponcompletingthestepsjustdescribed,wehaveaU-Bootsourcetreethatrepresentsastarting point. It probably will not even compile cleanly, and that should be our first step. At least the compilercangiveussomeguidanceonwheretostart. 7.4.3.EP405ProcessorInitialization ThefirsttaskthatyournewU-Bootportmustdocorrectlyistoinitializetheprocessorandthe memory(DRAM)subsystems.Afterreset,the405GPprocessorcoreisdesignedtofetchinstructions startingfrom0xFFFF_FFFC.Thecoreattemptstoexecutetheinstructionsfoundhere.Becausethisis thetopofthememoryrange,theinstructionfoundheremustbeanunconditionalbranchinstruction. Thisprocessorcoreisalsohard-codedtoconfiguretheupper2MBmemoryregionsothatitis accessible without programming the external bus controller, to which Flash memory is usually attached.Thisforcestherequirementtobranchtoalocationwithinthisaddressspacebecausethe processor is incapable of addressing memory anywhere else until our bootloader code initializes additionalmemoryregions.Wemustbranchtosomewhereatorabove0xFFE0_0000.Howdidwe knowallthis?Becausewereadthe405GPuser'smanual! The behavior of the 405GP processor core, as described in the previous paragraph, places requirementson thehardware designer toensurethat,onpower-up,nonvolatilememory(Flash)is mappedtotherequiredupper2MBmemoryregion.Certainattributesofthisinitialmemoryregion assumedefaultvaluesonreset.Forexample,thisupper2MBregionwillbeconfiguredfor256wait states,threecyclesofaddress-to-chipselectdelay,threecyclesofchipselecttooutputenabledelay, andsevencyclesofholdtime.[58]Thisallowsmaximumfreedomforthehardwaredesignertoselect appropriatedevicesormethodsofgettinginstructioncodetotheprocessordirectlyafterreset. We've already seen how the reset vector is installed to the top of Flash in Listing 7-2. When configuredforthe405GP,ourfirstlinesofcodewillbefoundinthefile.../cpu/ppc4xx/start.S.The U-Bootdevelopersintendedthiscodetobeprocessorgeneric.Intheory,thereshouldbenoneedfor board-specificcodeinthisfile.Youwillseehowthisisaccomplished. Wedon'tneedtounderstandPowerPCassemblylanguageinanydepthtounderstandthelogical flowinstart.S.Manyfrequentlyaskedquestions(FAQs)havebeenpostedtotheU-Bootmailinglist aboutmodifyinglow-levelassemblycode.Innearlyallcases,itisnotnecessarytomodifythiscode ifyouareportingtooneofthemanysupportedprocessors.Itismaturecode,withmanysuccessful portsrunningonit.Youneedtomodifytheboard-specificcode(atabareminimum)foryourport.If youfindyourselftroubleshootingormodifyingtheearlystartupassemblercodeforaprocessorthat hasbeenaroundforawhile,youaremostlikelyheadingdownthewrongroad. Listing7-6reproducesaportionofstart.Sforthe4xxarchitecture. Listing7-6.U-Boot4xxstartupcode ... #ifdefined(CONFIG_405GP)||defined(CONFIG_405CR)|| defined(CONFIG_405)||defined(CONFIG_405EP) /*---------------------------------*/ /*Clearandsetupsomeregisters.*/ /*---------------------------------*/ addir4,r0,0x0000 mtsprsgr,r4 mtsprdcwr,r4 mtesrr4/*clearExceptionSyndromeReg*/ mttcrr4/*clearTimerControlReg*/ mtxerr4/*clearFixed-PointExceptionReg*/ mtevprr4/*clearExceptionVectorPrefixReg*/ addir4,r0,0x1000/*setMEbit(MachineExceptions)*/ orisr4,r4,0x0002/*setCEbit(CriticalExceptions)*/ mtmsrr4/*changeMSR*/ addir4,r0,(0xFFFF-0x10000)/*setr4to0xFFFFFFFF(statusinthe*/ /*dbsrisclearedbysettingbitsto1)*/ mtdbsrr4/*clear/resetthedbsr*/ /*----------------------------------*/ /*InvalidateIandDcaches.EnableIcachefordefinedmemoryregions*/ /*tospeedthingsup.LeavetheDcachedisabledfornow.Itwillbe*/ /*enabled/leftdisabledlaterbasedonuserselectedmenuoptions.*/ /*BeawarethattheIcachemaybedisabledlaterbasedonthemenu*/ /*optionsaswell.SeemiscLib/main.c.*/ /*-------------------------------------*/ blinvalidate_icache blinvalidate_dcache /*--------------------------------------*/ /*Enabletwo128MBcachableregions.*/ /*-----------------------------------*/ addisr4,r0,0x8000 addir4,r4,0x0001 mticcrr4/*instructioncache*/ isync addisr4,r0,0x0000 addir4,r4,0x0000 mtdccrr4/*datacache*/ Thefirstcodetoexecuteinstart.Sforthe405GPprocessorstartsaboutathirdofthewayintothe source file, where a handful of processor registers are cleared or set to sane initial values. The instructionanddatacachesaretheninvalidated,andtheinstructioncacheisenabledtospeedupthe initial load. Two 128MB cacheable regions are set up, one at the high end of memory (the Flash region)andtheotheratthebottom(normallythestartofsystemDRAM).U-Booteventuallyiscopied toRAMinthisregionandexecutedfromthere.Thereasonforthisisperformance:Rawreadsfrom RAMareanorderofmagnitude(ormore)fasterthanreadsfromFlash.However,forthe4xxCPU, thereisanothersubtlereasonforenablingtheinstructioncache,asweshallsoondiscover. 7.4.4.Board-SpecificInitialization Thefirstopportunityforanyboard-specificinitializationcomesin.../cpu/ppc4xx/start.Sjustafter thecacheableregionshavebeeninitialized.Herewefindacalltoanexternalassemblerlanguage routinecalledext_bus_cntlr_init. blext_bus_cntlr_init/*Boardspecificbuscntrlinit*/ This routine is defined in .../board/ep405/init.S, in the new board-specific directory for our board.Itprovidesahookforveryearlyhardware-basedinitialization.Thisisoneofthefilesthathas beencustomizedforourEP405platform.Thisfilecontainstheboard-specificcodetoinitializethe 405GP'sexternalbuscontrollerforourapplication.Listing7-7containsthemeatofthefunctionality fromthisfile.Thisisthecodethatinitializesthe405GP'sexternalbuscontroller. Listing7-7.ExternalBusControllerInitialization .globlext_bus_cntlr_init ext_bus_cntlr_init: mflrr4/*savelinkregister*/ bl..getAddr ..getAddr: mflrr3/*get_this_address*/ mtlrr4/*restorelinkregister*/ addir4,0,14/*prefetch14cachelines...*/ mtctrr4/*...tofitthisfunction*/ /*cache(8x14=112instr)*/ ..ebcloop: icbtr0,r3/*prefetchcachelinefor[r3]*/ addir3,r3,32/*movetonextcacheline*/ bdnz..ebcloop/*continuefor14cachelines*/ /*---------------------------------------------------*/ /*DelaytoensureallaccessestoROMarecomplete*/ /*beforechangingbank0timings*/ /*200usecshouldbeenough.*/ /*200,000,000(cycles/sec)X.000200(sec)=*/ /*0x9C40cycles*/ /*---------------------------------------------------*/ addisr3,0,0x0 orir3,r3,0xA000/*ensure200usechavepassedt*/ mtctrr3 ..spinlp: bdnz..spinlp/*spinloop*/ /*----------------------------------------------------*/ /*Nowdotherealworkofthisfunction*/ /*MemoryBank0(FlashandSRAM)initialization*/ /*----------------------------------------------------*/ addir4,0,pb0ap/**ebccfga=pb0ap;*/ mtdcrebccfga,r4 addisr4,0,EBC0_B0AP@h/**ebccfgd=EBC0_B0AP;*/ orir4,r4,EBC0_B0AP@l mtdcrebccfgd,r4 addir4,0,pb0cr/**ebccfga=pb0cr;*/ mtdcrebccfga,r4 addisr4,0,EBC0_B0CR@h/**ebccfgd=EBC0_B0CR;*/ orir4,r4,EBC0_B0CR@l mtdcrebccfgd,r4 /*----------------------------------------------------*/ /*MemoryBank4(NVRAM&BCSR)initialization*/ /*----------------------------------------------------*/ addir4,0,pb4ap/**ebccfga=pb4ap;*/ mtdcrebccfga,r4 addisr4,0,EBC0_B4AP@h/**ebccfgd=EBC0_B4AP;*/ orir4,r4,EBC0_B4AP@l mtdcrebccfgd,r4 addir4,0,pb4cr/**ebccfga=pb4cr;*/ mtdcrebccfga,r4 addisr4,0,EBC0_B4CR@h/**ebccfgd=EBC0_B4CR;*/ orir4,r4,EBC0_B4CR@l mtdcrebccfgd,r4 blr/*return*/ TheexampleinListing7-7waschosenbecauseitistypicalofthesubtlecomplexitiesinvolved in low-level processor initialization. It is important to realize the context in which this code is running.ItisexecutingfromFlash,beforeanyDRAMisavailable.Thereisnostack.Thiscodeis preparing tomakefundamental changesto the controllerthatgovernsaccess totheveryFlashitis executing from. It is well documented for this particular processor that executing code from Flash whilemodifying theexternal bus controllertowhich the Flash is attachedcan leadtoerrantreads andaresultingprocessorcrash. Thesolutionisshowninthisassemblylanguageroutine.Startingatthelabel..getAddr,andfor the next seven assembly language instructions, the code essentially prefetches itself into the instruction cache, using the icbt instruction. When the entire subroutine has been successfully read intotheinstructioncache,itcanproceedtomaketherequiredchangestotheexternalbuscontroller withoutfearofacrashbecauseitisexecutingdirectlyfromtheinternalinstructioncache.Subtle,but clever! This is followed by a short delay to make sure all the requested i-cache reads have completed. Whentheprefetchanddelayhavecompleted,thecodeproceedstoconfigureMemoryBank0and Memory Bank 4 appropriately for our board. The values come from a detailed knowledge of the underlyingcomponentsandtheirinterconnectionontheboard.Theinterestedreadercanconsultthe "Suggestions for Additional Reading" at the end of the chapter for all the details of PowerPC assemblerandthe405GPprocessorfromwhichthisexamplewasderived. Consider making a change to this code without a complete understanding of what is happening here.Perhapsyouaddedafewlinesandincreaseditssizebeyondtherangethatwasprefetchedinto the cache. It would likely crash (worse, it might crash only sometimes), but stepping through this codewithadebuggerwouldnotyieldasingleclueastowhy. The next opportunity for board-specific initialization comes after a temporary stack has been allocated from the processor's data cache. This is the branch to initialize the SDRAM controller aroundline727of.../cpu/ppc4xx/start.S: blsdram_init Theexecutioncontextnowincludesastackpointerandsometemporarymemoryforlocaldata storagethatis,apartialCcontext,allowingthedevelopertouseCfortherelativelycomplextaskof setting up the system SDRAM controller and other initialization tasks. In our EP405 port, the sdram_init() code resides in .../board/ep405/ep405.c and was customized for this particular board andDRAMconfiguration.BecausethisboarddoesnotuseacommerciallyavailablememorySIMM, itisnotpossibletodeterminetheconfigurationoftheDRAMdynamically,likesomanyotherboards supportedbyU-Boot.Itishard-codedinsdram_init. Many off-the-shelf memory DDR modules have a SPD (Serial Presence Detect) PROM containing parameters defining the memory module. These parameters can be read under program controlviaI2Candcanbeusedasinputtodetermineproperparametersforthememorycontroller. U-Boot has support for this technique but might need to be modified to work with your specific board.ManyexamplesofitsusecanbefoundintheU-Bootsourcecode.Theconfigurationoption CONFIG_SPD_EEPROMenablesthisfeature.Youcangrepforthisoptiontofindexamplesofits use. 7.4.5.PortingSummary By now, you can appreciate some of the difficulties of porting a bootloader to a hardware platform. There is simply no substitute for a detailed knowledge of the underlying hardware. Of course,we'dliketominimizeourinvestmentintimerequiredforthistask.Afterall,weusuallyare notpaidbasedonhowwellweunderstandeveryhardwaredetailofagivenprocessor,butratheron ourabilitytodeliveraworkingsolutioninatimelymanner.Indeed,thisisoneoftheprimaryreasons open source has flourished. We just saw how easy it was to port U-Boot to a new hardware platformnot because we're world-class experts on the processor, but because many before us have donethebulkofthehardworkalready. Listing7-8isthecompletelistofnewormodifiedfilesthatcompletethebasicEP405portforUBoot.Ofcourse,iftherehadbeennewhardwaredevicesforwhichnosupportexistsinU-Boot,orif we were porting to a new CPU that is not yet supported in U-Boot, this would have been a much moresignificanteffort.Thepointtobemadehere,attheriskofsoundingredundant,isthatthereis simply no substitute for a detailed knowledge of both the hardware (CPU and subsystems) and the underlyingsoftware(U-Boot)tocompleteaportsuccessfullyinareasonabletimeframe.Ifyoustart theprojectfromthatframeofmind,youwillhaveasuccessfuloutcome. Listing7-8.NeworChangedFilesforU-BootEP405Port $diff-purNu-bootu-boot-ep405/|grep+++ +++u-boot-ep405/board/ep405/config.mk +++u-boot-ep405/board/ep405/ep405.c +++u-boot-ep405/board/ep405/ep405.h +++u-boot-ep405/board/ep405/flash.c +++u-boot-ep405/board/ep405/init.S +++u-boot-ep405/board/ep405/Makefile +++u-boot-ep405/board/ep405/u-boot.lds +++u-boot-ep405/include/config.h +++u-boot-ep405/include/config.mk +++u-boot-ep405/include/configs/EP405.h +++u-boot-ep405/include/ppc405.h +++u-boot-ep405/Makefile Recall that we derived all the files in the .../board/ep405 directory from another directory. Indeed,wedidn'tcreateanyfilesfromscratchforthisport.Weborrowedfromtheworkofothers andcustomizedwherenecessarytoachieveourgoals. 7.4.6.U-BootImageFormat NowthatwehaveaworkingbootloaderforourEP405board,wecanloadandrunprogramson it.Ideally,wewanttorunanoperatingsystemsuchasLinux.Todothis,weneedtounderstandthe image format that U-Boot requires. U-Boot expects a small header on the image file that identifies several attributes of the image. U-Boot uses the mkimage tool (part of the U-Boot source code) to buildthisimageheader. RecentLinuxkerneldistributionshavebuilt-insupportforbuildingimagesdirectlybootableby U-Boot.BoththeARMandPPCbranchesofthekernelsourcetreehavesupportforatargetcalled uImage. Let's look at the PPC case. The following snippet from the Linux kernel PPC makefile .../arch/ppc/boot/images/MakefilecontainstheruleforbuildingtheU-BoottargetcalleduImage: quiet_cmd_uimage=UIMAGE$@ cmd_uimage=$(CONFIG_SHELL)$(MKIMAGE)-Appc\ -Olinux-Tkernel-Cgzip-a00000000-e00000000\ -n'Linux-$(KERNELRELEASE)'-d$<$@ Ignoringthesyntacticalcomplexity,understandthatthisrulecallsashellscriptidentifiedbythe variable $(MKIMAGE). The shell script executes the U-Boot mkimage utility with the parameters shown.ThemkimageutilitycreatestheU-Bootheaderandprependsittothesuppliedkernelimage. Theparametersaredefinedasfollows: -ASpecifiesthetargetimagearchitecture -OSpeciesthetargetimageOSinthiscase,Linux -TSpecifiesthetargetimagetypeakernel,inthiscase -CSpecifiesthetargetimagecompressiontypehere,gzip -aSetstheU-Bootloadaddresstothevaluespecifiedinthiscase,0 -eSetstheU-Bootimageentrypointtothesuppliedvalue -nAtextfieldusedtoidentifytheimagetothehumanuser -dTheexecutableimagefiletowhichtheheaderisprepended SeveralU-Bootcommandsusethisheaderdatabothtoverifytheintegrityoftheimage(U-Boot alsoputsaCRCsignatureintheheader)andtoinstructvariouscommandswhattodowiththeimage. U-Boothasacommandcallediminfothatreadstheimageheaderanddisplaystheimageattributes from the target image. Listing 7-9 contains the results of loading a uImage (bootable Linux kernel imageformattedforU-Boot)totheEP405boardviaU-Boot'stftpbootcommandandexecutingthe iminfocommandontheimage. Listing7-9.U-BootiminfoCommand =>tftpboot400000uImage-ep405 ENETSpeedis100Mbps-FULLduplexconnection TFTPfromserver192.168.1.9;ourIPaddressis192.168.1.33 Filename'uImage-ep405'. Loadaddress:0x400000 Loading:##########done Bytestransferred=891228(d995chex) =>iminfo ##CheckingImageat00400000... ImageName:Linux-2.6.11.6 ImageType:PowerPCLinuxKernelImage(gzipcompressed) DataSize:891164Bytes=870.3kB LoadAddress:00000000 EntryPoint:00000000 VerifyingChecksum...OK => 7.5.OtherBootloaders Hereweintroducethemorepopularbootloaders,describewheretheymightbeused,andgivea summary of their features. This is not intended to be a thorough tutorial because to do so would requireabookofitsown.Theinterestedreadercanconsultthe"SuggestionsforAdditionalReading" attheendofthischapterforfurtherstudy. 7.5.1.Lilo The Linux Loader, or Lilo, was widely used in commercial Linux distributions for desktop PC platforms;assuch,ithasitsrootsintheIntelx86/IA32architecture.Lilohasseveralcomponents.It has a primary bootstrap program that lives on the first sector of a bootable disk drive.[59] The primaryloaderislimitedtoadisksectorsize,usually512bytes.Therefore,itsprimarypurposeis simply to load and pass control to a secondary loader. The secondary loader can span multiple partitionsanddoesmostoftheworkofthebootloader. Lilo is driven by a configuration file and utility that is part of the lilo executable. This configurationfilecanbereadorwrittentoonlyundercontrolofthehostoperatingsystem.Thatis, the configuration file is not referenced by the early boot code in either the primary or secondary loaders. Entries in the configuration file are read and processed by the lilo configuration utility during system installation or administration. Listing 7-10 is an example of a simple lilo.conf configurationfiledescribingatypicaldual-bootLinuxandWindowsinstallation. Listing7-10.ExampleLiloConfiguration:lilo.conf #Thisisthegloballiloconfigurationsection #Thesesettingsapplytoallthe"image"sections boot=/dev/hda timeout=50 default=linux #Thisdescribestheprimarykernelbootimage #Lilowilldisplayitwiththelabel'linux' image=/boot/myLinux-2.6.11.1 label=linux initrd=/boot/myInitrd-2.6.11.1.img read-only append="root=LABEL=/" #ThisisthesecondOSinadual-bootconfiguration #Thisentrywillbootasecondaryimagefrom/dev/hda1 other=/dev/hda1 optional label=that_other_os ThisconfigurationfileinstructstheLiloconfigurationutilitytousethemasterbootrecordofthe firstharddrive(/dev/hda).Itcontainsadelayinstructiontowaitfortheusertopressakeybeforethe timeout(5seconds,inthiscase).ThisgivesthesystemoperatorthechoicetoselectfromalistofOS imagestoboot.IfthesystemoperatorpressestheTabkeybeforethetimeout,Lilopresentsalistto choosefrom.Lilousesthelabeltagasthetexttodisplayforeachimage. Theimagesaredefinedwiththeimagetagintheconfigurationfile.Intheexamplepresentedin Listing 7-10, the primary (default) image is a Linux kernel image with a file name of myLinux2.6.11.1.Liloloadsthisimagefromtheharddrive.Itthenloadsasecondfiletobeusedasaninitial ramdisk.ThisisthefilemyInitrd-2.6.11.1.img.Liloconstructsakernelcommandlinecontainingthe string "root=LABEL=/" and passes this to the Linux kernel upon execution. This instructs Linux wheretogetitsrootfilesystemafterboot. 7.5.2.GRUB Many current commercial Linux distributions now ship with the GRUB bootloader. GRUB, or GRandUnifiedBootloader,isaGNUproject.IthasmanyenhancedfeaturesnotfoundinLilo.The biggest difference between GRUB and Lilo is GRUB's capability to understand file systems and kernelimageformats.Furthermore,GRUBcanreadandmodifyitsconfigurationatboottime.GRUB also supports booting across a network, which can be a tremendous asset in an embedded environment.GRUBoffersacommandlineinterfaceatboottimetomodifythebootconfiguration. LikeLilo,GRUBisdrivenbyaconfigurationfile.UnlikeLilo'sstaticconfigurationhowever,the GRUBbootloaderreadsthisconfigurationatboottime.Thismeansthattheconfiguredbehaviorcan bemodifiedatboottimefordifferentsystemconfigurations. Listing7-11isanexampleGRUBconfigurationfile.ThisistheconfigurationfilefromthePCon which this manuscript is being written. The GRUB configuration file is called grub.conf and is usuallyplacedinasmallpartitiondedicatedtostoringbootimages.Onthemachinefromwhichthis exampleistaken,thatdirectoryiscalled/boot. Listing7-11.ExampleGRUBConfigurationFile:grub.conf default=0 timeout=3 splashimage=(hd0,1)/grub/splash.xpm.gz titleFedoraCore2(2.6.9) root(hd0,1) kernel/bzImage-2.6.9roroot=LABEL=/rhgbproto=impsquiet initrd/initrd-2.6.9.img titleFedoraCore(2.6.5-1.358) root(hd0,1) kernel/vmlinuz-2.6.5-1.358roroot=LABEL=/rhgbquiet titleThatOtherOS rootnoverify(hd0,0) chainloader+1 GRUBfirstpresentstheuserwithalistofimagesthatareavailabletoboot.Thetitleentriesfrom Listing7-11aretheimagenamespresentedtotheuser.Thedefaulttagspecifieswhichimagetoboot ifnokeyshavebeenpressedinthetimeoutperiod,whichis3secondsinthisexample.Imagesare countedstartingfromzero. Unlike Lilo, GRUBcan actually readafilesystemonagivenpartitiontoloadan image from. Theroottagspecifiestherootpartitionfromwhichallfilenamesinthegrub.confconfigurationfile arerooted.Inthisexampleconfiguration,therootispartitionnumber1onthefirstharddiskdrive, specifiedasroot(hd0,1).Partitionsarenumberedfromzero;thisisthesecondpartitiononthefirst harddisk. Theimagesarespecifiedasfilenamesrelativetothespecifiedroot.InListing7-11,thedefault boot image is a Linux 2.6.9 kernel with a matching initial ramdisk image called initrd-2.6.9.img. NoticethattheGRUBsyntaxhasthekernelcommandlineparametersonthesamelineasthekernel filespecification. 7.5.3.StillMoreBootloaders Numerousotherbootloadershavefoundtheirwayintospecificniches.Forexample,Redbootis anotheropen-sourcebootloaderthatIntelandtheXScalecommunityhaveadoptedforuseonvarious evaluation boards based on the Intel IXP and PXA processor families. Micromonitor is in use by board vendors such as Cogent and others. YAMON has found popularity in MIPs circles.[60] LinuxBIOSisusedprimarilyinX86environments.Ingeneral,whenyouconsiderabootloader,you shouldconsidersomeimportantfactorsupfront: •Doesitsupportmychosenprocessor? •Hasitbeenportedtoaboardsimilartomyown? •DoesitsupportthefeaturesIneed? •DoesitsupportthehardwaredevicesIintendtouse? •IstherealargecommunityofuserswhereImightgetsupport? •ArethereanycommercialvendorsfromwhichIcanpurchasesupport? These are some of the questions you must answer when considering what bootloader to use in yourembeddedproject.Unlessyouaredoingsomethingonthe"bleedingedge"oftechnologyusinga brand-newprocessor,youarelikelytofindthatsomeonehasalreadydonethebulkofthehardwork inportingabootloadertoyourchosenplatform.Usetheresourcesattheendofthischaptertohelp makeyourfinaldecisions. 7.6.ChapterSummary • The bootloader's role in an embedded system cannot be overstated. It is the first piece of softwarethattakescontroluponapplyingpower. •Thischapterexaminedtheroleofthebootloaderanddiscoveredthelimitedexecutioncontext inwhichabootloadermustexist. • Das U-Boot has become a popular universal bootloader for many processor architectures. It supportsalargenumberofprocessors,referencehardwareplatforms,andcustomboards. •U-Bootisconfiguredusingaseriesofconfigurationvariablesinaboard-specificheaderfile. AppendixB,containsalistofallthestandardU-BootcommandsetssupportedinarecentU-Boot release. •PortingU-Boottoanewboardbasedonasupportedprocessorisrelativelystraightforward.In thischapter,wewalkedthroughthestepsofatypicalporttoaboardwithsimilarsupportinU-Boot. •Thereisnosubstitutefordetailedknowledgeofyourprocessorandhardwareplatformwhen bootloadermodificationorportingmustbeaccomplished. •Webrieflyintroducedadditionalbootloadersinusetodaysoyoucanmakeaninformedchoice foryourparticularrequirements. 7.6.1.SuggestionsforAdditionalReading ApplicationNote:IntroductiontoSynchronousDRAM MaxwellTechnologies www.maxwell.com/pdf/me/app_notes/Intro_to_SDRAM.pdf UsingLD,theGNUlinker FreeSoftwareFoundation www.gnu.org/software/binutils/manual/ld-2.9.1/ld.html TheDENXU-BootandLinuxGuide(DLUG)forTQM8xxL WolfgangDenxetal.,DenxSoftwareEngineering www.denx.de/twiki/bin/view/DULG/Manual RFC793,"TrivialFileTransferProtocol" TheInternetEngineeringTaskForce www.ietf.org/rfc/rfc793.txt RFC951,"BootstrapProtocol" TheInternetEngineeringTaskForce www.ietf.org/rfc/rfc951.txt RFC1531,"DynamicHostControlProtocol" TheInternetEngineeringTaskForce www.ietf.org/rfc/rfc1531.txt PowerPC405GPEmbeddedProcessoruser'smanual InternationalBusinessMachines,Inc. ProgrammingEnvironmentsManualfor32-bitImplementationsofthePowerPCArchitecture FreescaleSemiconductor,Inc. LiloBootloader www.tldp.org/HOWTO/LILO.html GRUBBootloader www.gnu.org/software/grub/ Chapter8.DeviceDriverBasics One of the more challenging aspects of system design is partitioning functionality in a rational manner.ThefamiliardevicedrivermodelfoundinUNIXandLinuxprovidesanaturalpartitioningof functionality between your application code and hardware or kernel devices. In this chapter, we develop an understanding of this model and the basics of Linux device driver architecture. After reading this chapter, you will have a solid foundation for continuing your study of device drivers usingoneoftheexcellenttextslistedattheendofthischapter. ThischapterbeginsbypresentingLinuxdevicedriverconceptsandthebuildsystemfordrivers withinthekernelsourcetree.WeexaminetheLinuxdevicedriverarchitectureandpresentasimple working example driver. We introduce the user space utilities for loading and unloading kernel modules.[61] We present a simple application to illustrate the interface between applications and device drivers. We conclude this chapter with a discussion of device drivers and the GNU Public License. 8.1.DeviceDriverConcepts Manyexperiencedembeddeddevelopersstruggleatfirstwiththeconceptsofdevicedriversina virtualmemoryoperatingsystem.Thisisbecausemanypopularlegacyreal-timeoperatingsystems donothaveasimilararchitecture.Theintroductionofvirtualmemoryandkernelspaceversususer spacefrequentlyintroducescomplexitythatisnotfamiliartoexperiencedembeddeddevelopers. Oneofthefundamentalpurposesofadevicedriveristoisolatetheuser'sprogramsfromready access to critical kernel data structures and hardware devices. Furthermore, a well-written device driver hides the complexity and variability of the hardware device from the user. For example, a program that wants to write data to the hard disk need not care if the disk drive uses 512-byte or 1024-byte sectors. The user simply opens a file and issues a write command. The device driver handles the details and isolates the user from the complexities and perils of hardware device programming.Thedevicedriverprovidesaconsistentuserinterfacetoalargevarietyofhardware devices. It provides the basis for the familiar UNIX/Linux convention that everything must be representedasafile. 8.1.1.LoadableModules Unlike some other operating systems, Linux has the capability to add and remove kernel componentsatruntime.Linuxisstructuredasamonolithickernelwithawell-definedinterfacefor addingandremovingdevicedrivermodulesdynamicallyafterboottime.Thisfeaturenotonlyadds flexibilitytotheuser,butithasproveninvaluabletothedevicedriverdevelopmenteffort.Assuming thatyourdevicedriverisreasonablywellbehaved,youcaninsertandremovethedevicedriverfrom a running kernel at will during the development cycle instead of rebooting the kernel every time a changeoccurs. Loadablemoduleshaveparticularimportancetoembeddedsystems.Loadablemodulesenhance fieldupgradecapabilities;themoduleitselfcanbeupdatedinalivesystemwithouttheneedfora reboot. Modules can be stored on media other than the root (boot) device, which can be space constrained. Ofcourse,devicedriverscanalsobestaticallycompiledintothekernel,and,formanydrivers, thisiscompletelyappropriate.Consider,forexample,akernelconfiguredtomountarootfilesystem from a network-attached NFS server. In this scenario, you configure the network-related drivers (TCP/IPandthenetworkinterfacecarddriver)tobecompiledintothemainkernelimagesotheyare available during boot for mounting the remote root file system. You can use the initial ramdisk functionality as described in Chapter 6, "System Initialization," as an alternative to having these drivers compiled statically as part of the kernel proper. In this case, the necessary modules and a scripttoloadthemwouldbeincludedintheinitialramdiskimage. Loadablemodulesareinstalledafterthekernelhasbooted.Startupscriptscanloaddevicedriver modules,andmodulescanalsobe"demandloaded"whenneeded.Thekernelhasthecapabilityto requestamodulewhenaserviceisrequestedthatrequiresaparticularmodule. Terminology has never been standardized when discussing kernel modules. Many terms have beenandcontinuetobeusedinterchangeablywhendiscussingloadablekernelmodules.Throughout thisandlaterchapters,thetermsdevicedriver,loadablekernelmodule(LKM),loadablemodule, andmoduleareallusedtodescribealoadablekerneldevicedrivermodule. 8.1.2.DeviceDriverArchitecture ThebasicLinuxdevicedrivermodelisfamiliartoUNIX/Linuxsystemdevelopers.Althoughthe devicedrivermodelcontinuestoevolve,somefundamentalconstructshaveremainednearlyconstant over the course of UNIX/Linux evolution. Device drivers are broadly classified into two basic categories: character devices and block devices. Character devices can be thought of as serial streamsofsequentialdata.Examplesofcharacterdevicesincludeserialportsandkeyboards.Block devices are characterized by the capability to read and write blocks of data to and from random locationsonanaddressablemedium.Examplesofblockdevicesincludeharddrivesandfloppydisk drives. 8.1.3.MinimalDeviceDriverExample Because Linux supports loadable device drivers, it is relatively easy to demonstrate a simple devicedriverskeleton.Listing8-1illustratesaloadabledevicedrivermodulethatcontainsthebare minimumstructuretobeloadedandunloadedbyarunningkernel. Listing8-1.MinimalDeviceDriver /*ExampleMinimalCharacterDeviceDriver*/ #include<linux/module.h> staticint__inithello_init(void){ printk("HelloExampleInit\n"); return0; } staticvoid__exithello_exit(void){ printk("HelloExampleExit\n"); } module_init(hello_init); module_exit(hello_exit); MODULE_AUTHOR("ChrisHallinan"); MODULE_DESCRIPTION("HelloWorldExample"); MODULE_LICENSE("GPL"); TheskeletaldriverinListing8-1containsenoughstructureforthekerneltoloadandunloadthe driver, and to invoke the initialization and exit routines. Let's look at how this is done because it illustratessomeimportanthigh-levelconceptsthatareusefulfordevicedriverdevelopment. A device driver is a special kind of binary module. Unlike a stand-alone binary executable application, a device driver cannot be simply executed from a command prompt. The 2.6 kernel seriesrequiresthatthebinarybeinaspecial"kernelobject"format.Whenproperlybuilt,thedevice driverbinarymodulecontainsa.kosuffix.Thebuildstepsandcompileroptionsrequiredtocreate the.komoduleobjectcanbequitecomplex.Hereweoutlineasetofstepstoharnessthepowerof theLinuxkernelbuildsystemwithoutrequiringyoutobecomeanexpertinit,whichisbeyondthe scopeofthisbook. 8.1.4.ModuleBuildInfrastructure A device driver must be compiled against the kernel on which it will execute. Although it is possibletoloadandexecutekernelmodulesbuiltagainstadifferentkernelversion,itisriskytodo sounlessyouarecertainthatthemoduledoesnotrelyonanyfeaturesofyournewkernel.Theeasiest way to do this is to build the module within the kernel's own source tree. This ensures that as the developer changes the kernel configuration, his custom driver is automatically rebuilt with the correctkernelconfiguration.Itiscertainlypossibletobuildyourdriversoutsideofthekernelsource tree. However, in this case, you are responsible for making sure that your device driver build configuration stays in sync with the kernel you want to run your driver on. This typically includes compilerswitches,locationofkernelheaderfiles,andkernelconfigurationoptions. FortheexampledriverintroducedinListing8-1,thefollowingchangesweremadetothestock Linuxkernelsourcetreetoenablebuildingthisexampledriver.Weexplaineachstepindetail. 1. Starting from the top-level Linux source directory, create a directory under .../drivers/char calledexamples. 2.Addamenuitemtothekernelconfigurationtoenablebuildingexamplesandtospecifybuilt-in orloadablekernelmodule. 3.Add the new examples subdirectory to the .../drivers/char/Makefile conditional on the menu itemcreatedinstep2. 4.Createa makefile forthenewexamplesdirectory,andaddthehello1.omoduleobjecttobe compiledconditionalonthemenuitemcreatedinstep2. 5.Finally,createthedriverhello1.csourcefilefromListing8.1. Adding the examples directory under the .../drivers/char subdirectory is self-explanatory. After this directory is created, two files are created in this directory: the module source file itself from Listing8-1andthemakefilefortheexamplesdirectory.Themakefileforexamplesisquitetrivial.It willcontainthissingleline: obj-$(CONFIG_EXAMPLES)+=hello1.o Adding the menu item to the kernel configuration utility is a little more involved. Listing 8-2 containsapatchthat,whenappliedtothe.../drivers/char/KconfigfilefromarecentLinuxrelease, addstheconfigurationmenuitemtoenableourexamplesconfigurationoption.Forthosereadersnot familiarwiththediff/patchformat,eachlineinListing8-1precededbyasingleplus(+)character isinsertedinthefilebetweentheindicatedlines(thosewithouttheleading+character). Listing8-2.KconfigPatchforExamples diff-u~/base/linux-2.6.14/drivers/char/Kconfig./drivers/char/Kconfig ---~/base/linux-2.6.14/drivers/char/Kconfig +++./drivers/char/Kconfig @@-4,6+4,12@@ menu"Characterdevices" +configEXAMPLES +tristate"EnableExamples" +defaultM +---help--+Enablecompilationoptionfordriverexamples + configVT bool"Virtualterminal"ifEMBEDDED selectINPUT WhenappliedtoKconfiginthe.../drivers/charsubdirectoryofarecentLinuxkernel,thispatch resultsinanewkernelconfigurationoptioncalledCONFIG_EXAMPLES.Asareminderfromour discussiononbuildingtheLinuxkernelinChapter4,"TheLinuxKernelADifferentPerspective,"the configurationutilityisinvokedasfollows(thisexampleassumestheARMarchitecture): $makeARCH=ARMCROSS_COMPILE=xscale_be-gconfig Aftertheconfigurationutilityisinvokedusingacommandsimilartothepreviousone,ournew EnableExamplesconfigurationoptionappearsundertheCharacterdevicesmenu,asindicatedinthe patch.Becauseitisdefinedastypetristate,thekerneldevelopercanchoosefromthreechoices: (N)No.Donotcompileexamples. (Y)Yes.Compileexamplesandlinkwithfinalkernelimage. (M)Module.Compileexamplesasdynamicallyloadablemodule. Figure8-1showstheresultinggconfigscreenwiththenewconfigurationoptionadded.Thedash (-)inthecheckboxselects(M)odule,asindicatedintheMcolumnontheright.Acheckmarkinthe checkboxselects(Y)es,indicatingthatthedrivermoduleshouldbecompiledaspartofthekernel proper.Anemptycheckboxindicatesthattheoptionisnotselected. Figure8-1.KernelconfigurationwithExamplesmodule Now that we have added the configuration option to enable compiling our examples device driver module, we need to modify the makefile in .../drivers/char to instruct the build system to descend into our new examples subdirectory if the configuration option CONFIG_EXAMPLES is present in our configuration. Listing 8-3 contains the patch for this against the makefile in a recent Linuxrelease. Listing8-3.MakefilePatchforExamples diff-u~/base/linux-2.6.14/drivers/char/Makefile./drivers/char/Makefile ---~/base/linux-2.6.14/drivers/char/Makefile +++./drivers/char/Makefile @@-88,6+88,7@@ obj-$(CONFIG_DRM)+=drm/ obj-$(CONFIG_PCMCIA)+=pcmcia/ obj-$(CONFIG_IPMI_HANDLER)+=ipmi/ +obj-$(CONFIG_EXAMPLES)+=examples/ obj-$(CONFIG_HANGCHECK_TIMER)+=hangcheck-timer.o ThepatchinListing8-3addsthesingleline(precededbythe+character)tothemakefilefound in .../drivers/char. The additional lines of context are there so that the patch utility can determine where to insert the new line. Our new examples directory was added to the end of the list of directoriesalreadybeingsearchedinthismakefile,whichseemedlikealogicalplacetoputit.Other thanforconsistencyandreadability,thelocationisirrelevant. Havingcompletedthestepsinthissection,theinfrastructureisnowinplacetobuildtheexample devicedriver.Thebeautyofthisapproachisthatthedriverisbuiltautomaticallywheneverakernel buildisinvoked.AslongastheconfigurationoptiondefinedinListing8-3isselected(eitherMor Y),thedrivermoduleisincludedinthebuild. Building for an arbitrary ARM system, the command line for building modules might look like this: $makeARCH=armCROSS_COMPILE=xscale_be-modules Listing8-4showsthebuildoutputafteratypicaleditingsessiononthemodule(allothermodules havealreadybeenbuiltinthiskernelsourcetree.) Listing8-4.ModuleBuildOutput $makeARCH=armCROSS_COMPILE=xscale_be-modules CHKinclude/linux/version.h make[1]:'arch/arm/kernel/asm-offsets.s'isuptodate. make[1]:'include/asm-arm/mach-types.h'isuptodate. CC[M]drivers/char/examples/hello1.o Buildingmodules,stage2. MODPOST LD[M]drivers/char/examples/hello1.ko 8.1.5.InstallingYourDeviceDriver Now that this driver is built, we can load and unload it on a running kernel to observe its behavior.Beforewecanloadthemodule,weneedtocopyittoanappropriatelocationonourtarget system.Althoughwecouldputitanywherewewant,aconventionisinplaceforkernelmodulesand wheretheyarepopulatedonarunningLinuxsystem.Aswithmodulecompilation,itiseasiesttolet the kernel build system do that for us. The makefile target modules_install automatically places modulesinthesysteminalogicallayout.Yousimplyneedtosupplythedesiredlocationasaprefix tothedefaultpath. In a standard Linux workstation installation, you might already know that the device driver modules live in /lib/modules/<kernel-version>/... ordered in a manner similar to the device driver directoryhierarchyintheLinuxkerneltree.[62]The<kernel-version>stringisproducedbyexecuting thecommanduname-ronyourtargetLinuxsystem.Ifyoudonotprovideaninstallationprefixtothe kernelbuildsystem,bydefault,yourmodulesareinstalledinyourownworkstation's/lib/modules/... directory.Thisisprobablynotwhatyouhadintended.Youcanpointtoatemporarylocationinyour home directory and manually copy the modules to your target's file system. Alternatively, if your targetembeddedsystemusesNFSrootmounttoadirectoryonyourlocaldevelopmentworkstation, you can install the modules directly to the target file system. The following example assumes the latter. $makeARCH=armCROSS_COMPILE=xscale_be-\ INSTALL_MOD_PATH=/home/chris/sandbox/coyote-target\ modules_install This places all your modules in the directory coyote-target, which on this example system is exportedviaNFSandmountedasrootonthetargetsystem.[63] 8.1.6.LoadingYourModule Havingcompletedallthestepsnecessary,wearenowinapositiontoloadandtestthedevice driver module. Listing 8-5 shows the output resulting from loading and subsequently unloading the devicedriverontheembeddedsystem. Listing8-5.LoadingandUnloadingaModule $modprobehello1<<<Loadthedriver HelloExampleInit $modprobe-rhello1<<<Unloadthedriver HelloExampleExit $ YoushouldbeabletocorrelatetheoutputwithourdevicedriversourcecodefoundinListing81.Themoduledoesnoworkotherthanprintingmessagestothekernellogsystemviaprintk(),which weseeonourconsole.[64]When themoduleisloaded,themodule-initializationfunction is called. We specify the initialization function that will be executed on module insertion using the module_init()macro.Wedeclareditasfollows: module_init(hello_init); Inourinitializationfunction,wesimplyprinttheobligatoryhellomessageandreturn.Inareal devicedriver,thisiswhereyouwouldperformanyinitialresourceallocationandhardwaredevice initialization.Inasimilarfashion,whenweunloadthemodule(usingthemodprobe-rcommand),our module exit routine is called. As shown in Listing 8-1, the exit routine is specified using the module_exit()macro. That'sallthereistoaskeletaldevicedrivercapableofliveinsertioninanactualkernel.Inthe sectionstofollow,weintroduceadditionalfunctionalitytoourloadabledevicedrivermodulethat illustrateshowauserspaceprogramwouldinteractwithadevicedrivermodule. 8.2.ModuleUtilities WehadabriefintroductiontomoduleutilitiesinListing8-5.Thereweusedthemoduleutility modprobe to insert and remove a device driver module from a Linux kernel. A number of small utilitiesareusedtomanagedevicedrivermodules.Thissectionintroducesthem.Youareencouraged to refer to the man page for each utility, for complete details. In fact, those interested in a greater knowledge of Linux loadable modules should consult the source code for these utilities. Section 8.6.1,"SuggestionsforAdditionalReading"attheendofthischaptercontainsareferenceforwhere theycanbefound. 8.2.1.insmod The insmod utility is the simplest way to insert a module into a running kernel. You supply a completepathname,andinsmoddoesthework.Forexample: $insmod/lib/modules/2.6.14/kernel/drivers/char/examples/hello1.ko This loads the module hello1.ko into the kernel. The output would be the same as shown in Listing8-5namely,theHellomessage.Theinsmodutilityisasimpleprogramthatdoesnotrequire oracceptanyoptions.Itrequiresafullpathnamebecauseithasnologicforsearchingforthemodule. Most often, you will use modprobe, described shortly, because it has many more features and capabilities. 8.2.2.ModuleParameters Manydevicedrivermodulescanacceptparameterstomodifytheirbehavior.Examplesinclude enablingdebugmode,settingverbosereporting,orspecifyingmodule-specificoptions.Theinsmod utilityacceptsparameters(alsocalledoptionsinsomecontexts)byspecifyingthemafterthemodule name.Listing8-6showsourmodifiedhello1.cexample,addingasinglemoduleparametertoenable debugmode. Listing8-6.ExampleDriverwithParameter /*ExampleMinimalCharacterDeviceDriver*/ #include<linux/module.h> staticintdebug_enable=0;/*Addeddriverparameter*/ module_param(debug_enable,int,0);/*andthese2lines*/ MODULE_PARM_DESC(debug_enable,"Enablemoduledebugmode."); staticint__inithello_init(void){ /*Nowprintvalueofnewmoduleparameter*/ printk("HelloExampleInit-debugmodeis%s\n",debug_enable?"enabled":"disabled"); return0; } staticvoid__exithello_exit(void){ printk("HelloExampleExit\n"); } module_init(hello_init); module_exit(hello_exit); MODULE_AUTHOR("ChrisHallinan"); MODULE_DESCRIPTION("HelloWorldExample"); MODULE_LICENSE("GPL"); Three lines have been added to our example device driver module. The first declares a static integertoholdourdebugflag.Thesecondlineisamacrodefinedin.../include/linux/moduleparam.h thatregistersthemoduleparameterwiththekernelmodulesubsystem.Thethirdnewlineisamacro that registers a string description associated with the parameter with the kernel module subsystem. Thepurposeofthiswillbecomeclearwhenweexaminethemodinfocommandlaterinthischapter. Ifwenowuseinsmodtoinsertourexamplemodule,andaddthedebug_enableoption,weshould seetheresultingoutput,basedonourmodifiedhello1.cmoduleinListing8-6. $insmod/lib/modules/.../examples/hello1.kodebug_enable=1 HelloExampleInit-debugmodeisenabled Or,ifweomittheoptionalmoduleparameter: $insmod/lib/modules/.../examples/hello1.ko HelloExampleInit-debugmodeisdisabled 8.2.3.lsmod Thelsmodutilityisalsoquitetrivial.Itsimplydisplaysaformattedlistofthemodulesthatare inserted into the kernel. Recent versions take no parameters and simply format the output of /proc/modules.[65]Listing8-7isanexampleoftheoutputfromlsmod. Listing8-7.lsmodExampleOutputFormat $lsmod ModuleSizeUsedby ext31210960 jbd496561ext3 loop127120 hello114120 $ Notice the rightmost column labeled Used by. This column indicates that the device driver module is in use and shows the dependency chain. In this example, the jbd module (journaling routines for journaling file systems) is being used by the ext3 module, the default journaling file systemformanypopularLinuxdesktopdistributions.Thismeansthattheext3devicedriverdepends onthepresenceofjbd. 8.2.4.modprobe Thisiswheretheclevernessofmodprobecomesintoplay.InListing8-7,weseetherelationship betweentheext3andjbdmodules.Theext3moduledependsonthejbdmodule.Themodprobeutility can discover this relationship and load the dependent modules in the proper order. The following commandloadsboththejbd.koandext3.kodrivermodules: $modprobeext3 The modprobe utility has several command line options that control its behavior. As we saw earlier,modprobecanbeusedtoremovemodules,includingthemodulesuponwhichagivenmodule depends.Hereisanexampleofmoduleremovalthatremovesbothjbd.koandext3.ko: $modprobe-rext3 The modprobe utility is driven by a configuration file called modprobe.conf. This enables a system developer to associate devices with device drivers. For a simple embedded system, modprobe.conf might be empty or might contain very few lines. The modprobe utility is compiled with a set of default rules that establish the defaults in the absence of a valid modprobe.conf. Invokingmodprobewithonlythe-coptiondisplaysthesetofdefaultrulesusedbymodprobe. Listing8-8representsatypicalmodprobe.conf,whichmightbefoundonasystemcontainingtwo Ethernetinterfaces;oneisawirelessadapterbasedonthePrism2chipset,andtheotherisatypical PCIEthernetcard.ThissystemalsocontainsasoundsubsystembasedonanintegratedIntelsound chipset. Listing8-8.Typicalmodprobe.confFile $cat/etc/modprobe.conf aliaseth1orinoci_pci optionseth1orinoco_debug=9 aliaseth0e100 aliassnd-card-0snd-intel8x0 optionssnd-card-0index=0 $ When the kernel boots and discovers the wireless chipset, this configuration file instructs modprobetoloadtheorinoco_pcidevicedriver,boundtokerneldeviceeth1,andpasstheoptional moduleparameterorinoco_debug=9tothedevicedriver.Thesameactionistakenupondiscoveryof the sound card hardware. Notice the optional parameters associated with the sound driver sndintel8x0. 8.2.5.depmod Howdoesmodprobeknowaboutthedependenciesofagivenmodule?Thedepmodutilityplays akeyroleinthisprocess.Whenmodprobeisexecuted,itsearchesforafilecalledmodules.depin the same location where the modules are installed. The depmod utility creates this moduledependencyfile. This file contains a list of all the modules that the kernel build system is configured for, along withdependencyinformationforeach.Itisasimplefileformat:Eachdevicedrivermoduleoccupies one line in the file. If the module has dependencies, they are listed in order following the module name. For example, from Listing 8-7, we saw that the ext3 module had a dependency on the jbd module.Thedependencylineinmodules.depwouldlooklikethis: ext3.ko:jbd.ko Inactualpractice,eachmodulenameisprecededbyitsabsolutepathinthefilesystem,toavoid ambiguity. We have omitted the path information for readability. A more complicated dependency chain,suchassounddrivers,mightlooklikethis: snd-intel8x0.ko:snd-ac97-codec.kosnd-pcm.kosnd-timer.ko\ snd.kosoundcore.kosnd-page-alloc.ko Again,wehaveremovedtheleadingpathcomponentsforreadability.Eachmodulefilenamein themodules.depfileisanabsolutefilename,withcompletepathinformation,andexistsonasingle line.Thepreviousexamplehasbeentruncatedtotwolines,tofitinthespaceonthispage. Normally,depmodisrunautomaticallyduringakernelbuild.However,inacross-development environment,youmusthaveacross-versionofdepmodthatknowshowtoreadthemodulesthatare compiledinthenativeformatofyourtargetarchitecture.Alternatively,mostembeddeddistributions have a method and init script entries to run depmod on each boot, to guarantee that the module dependenciesarekeptup-to-date. 8.2.6.rmmod This utility is also quite trivial. It simply removes a module from a running kernel. Pass it the modulenameasaparameter.Thereisnoneedtoincludeapathnameorfileextension.Forexample: $rmmodhello1 HelloExampleExit Theonlyinterestingpointtounderstandhereisthatwhenyouusermmod,itexecutesthemodule's *_exit()function,asshowninthepreviousexample,fromourhello1.cexampleofListings8-1and86. It should be noted that, unlike modprobe, rmmod does not remove dependent modules. Use modprobe-rforthis. 8.2.7.modinfo YoumighthavenoticedthelastthreelinesoftheskeletaldriverinListing8-1,andlaterinListing 8-6.Thesemacrosaretheretoplacetagsinthebinarymoduletofacilitatetheiradministrationand management.Listing8-9istheresultofmodinfoexecutedonourhello1.komodule. Listing8-9.modinfoOutput $modinfohello1 filename:/lib/modules/.../char/examples/hello1.ko author:ChrisHallinan description:HelloWorldExample license:GPL vermagic:2.6.14ARMv5gcc-3.3 depends: parm:debug_enable:Enablemoduledebugmode.(int) $ Thefirstfieldisobvious:Itisthefullfilenameofthedevicedrivermodule.Forreadabilityin this listing, we have truncated the path again. The next lines are a direct result of the descriptive macrosfoundattheendofListing8-6namely,thefilename,author,andlicenseinformation.These are simply tags for use by the module utilities and do not affect the behavior of the device driver itself.Youcanlearnmoreaboutmodinfofromitsmanpageandthemodinfosourceitself. Oneveryusefulfeatureofmodinfoistolearnwhatparametersthemodulesupports.FromListing 8-9,youcanseethatthismodulesupportsjustoneparameter.ThiswastheoneweaddedinListing 8-6,debug_enable.Thelistinggivesthename,type(inthiscase,anint),anddescriptivetextfieldwe enteredwiththeMODULE_PARM_DESC()macro.Thiscanbeveryhandy,especiallyformodules inwhichyoumightnothaveeasyaccesstothesourcecode. 8.3.DriverMethods We'vecoveredmuchgroundinourshorttreatmentofmoduleutilities.Intheremainingsectionsof thischapter,wedescribethebasicmechanismforcommunicatingwithadevicedriverfromauser spaceprogram(yourapplicationcode). Wehaveintroducedthetwofundamentalmethodsresponsibleforone-timeinitializationandexit processingofthemodule.RecallfromListing8-1thatthesearemodule_init()andmodule_exit().We discoveredthattheseroutinesareinvokedatthetimethemoduleisinsertedintoorremovedfroma runningkernel.Nowweneedsomemethodstointerfacewithourdevicedriverfromourapplication program.Afterall,twoofthemoreimportantreasonsweusedevicedriversaretoisolatetheuser fromtheperilsofwritingcodeinkernelspaceandtopresentaunifiedmethodtocommunicatewith hardwareorkernel-leveldevices. 8.3.1.DriverFileSystemOperations Afterthedevicedriverisloadedintoalivekernel,thefirstactionwemusttakeistopreparethe driver for subsequent operations. The open() method is used for this purpose. After the driver has beenopened,weneedroutinesforreadingandwritingtothedriver.Arelease()routineisprovided tocleanupafteroperationswhencomplete(basically,aclosecall).Finally,aspecialsystemcallis providedfornonstandardcommunicationtothedriver.Thisiscalledioctl().Listing8-10addsthis infrastructuretoourexampledevicedriver. Listing8-10.AddingFileSystemOpstoHello.c #include<linux/module.h> #include<linux/fs.h> #defineHELLO_MAJOR234 staticintdebug_enable=0; module_param(debug_enable,int,0); MODULE_PARM_DESC(debug_enable,"Enablemoduledebugmode."); structfile_operationshello_fops; staticinthello_open(structinode*inode,structfile*file){ printk("hello_open:successful\n"); return0; } staticinthello_release(structinode*inode,structfile*file){ printk("hello_release:successful\n"); return0; } staticssize_thello_read(structfile*file,char*buf,size_tcount,loff_t*ptr){ printk("hello_read:returningzerobytes\n"); return0; } staticssize_thello_write(structfile*file,constchar*buf,size_tcount,loff_t*ppos){ printk("hello_read:acceptingzerobytes\n"); return0; } staticinthello_ioctl(structinode*inode,structfile*file,unsignedintcmd,unsignedlongarg){ printk("hello_ioctl:cmd=%ld,arg=%ld\n",cmd,arg); return0; } staticint__inithello_init(void){ intret; printk("HelloExampleInit-debugmodeis%s\n",debug_enable?"enabled":"disabled"); ret=register_chrdev(HELLO_MAJOR,"hello1",&hello_fops); if(ret<0){ printk("Errorregisteringhellodevice\n"); gotohello_fail1; } printk("Hello:registeredmodulesuccessfully!\n"); /*Initprocessinghere...*/ return0; hello_fail1: returnret; } staticvoid__exithello_exit(void){ printk("HelloExampleExit\n"); } structfile_operationshello_fops={ owner:THIS_MODULE, read:hello_read, write:hello_write, ioctl:hello_ioctl, open:hello_open, release:hello_release, }; module_init(hello_init); module_exit(hello_exit); MODULE_AUTHOR("ChrisHallinan"); MODULE_DESCRIPTION("HelloWorldExample"); MODULE_LICENSE("GPL"); Thisexpandeddevicedriverexampleincludesmanynewlines.Fromthetop,we'vehadtoadda new kernel header file to get the definitions for the file system operations. We've also defined a majornumberforourdevicedriver.(Notetodevicedriverauthors:Thisisnottheproperwayto allocate a device driver major number. Refer to the Linux kernel documentation (.../Documentation/devices.txt) or one of the excellent texts on device drivers for guidance on the allocationofmajordevicenumbers.Forthissimpleexample,wesimplychooseonethatweknow isn'tinuseonoursystem.) Next we see definitions for four new functions, our open, close, read, and write methods. In keepingwithgoodcodingpractices,we'veadoptedaconsistentnamingschemethatwillnotcollide withanyothersubsystemsinthekernel.Ournewmethodsarecalledhello_open(),hello_release(), hello_read(), and hello_write(), respectively. For purposes of this simple exercise, they are do- nothingfunctionsthatsimplyprintamessagetothekernellogsubsystem. Noticethatwe'vealsoaddedanewfunctioncalltoourhello_init()routine.Thislineregisters ourdevicedriverwiththekernel.Withthatregistrationcall,wepassastructurecontainingpointers to the required methods. The kernel uses this structure, of type struct file_operations, to bind our specific device functions with the appropriate requests from the file system. When an application opens a device represented by our device driver and requests a read() operation, the file system associates that generic read() request with our module's hello_read() function. The following sectionsexaminethisprocessindetail. 8.3.2.DeviceNodesandmknod To understand how an application binds its requests to a specific device represented by our devicedriver,wemustunderstandtheconceptofadevicenode.Adevicenodeisaspecialfiletype in Linux that represents a device. Virtually all Linux distributions keep device nodes in a common location(specifiedbytheFilesystemHierarchyStandard[66]),inadirectorycalled/dev.Adedicated utilityisusedtocreateadevicenodeonafilesystem.Thisutilityiscalledmknod. Anexampleofnodecreationisthebestwaytoillustrateitsfunctionalityandtheinformationit conveys. In keeping with our simple device driver example, let's create the proper device node to exerciseit: $mknod/dev/hello1c2340 Afterexecutingthiscommandonourtargetembeddedsystem,weendupwithanewfilecalled /dev/hello1thatrepresentsourdevicedrivermodule.Ifwelistthisfiletotheconsole,itlookslike this: $ls-l/dev/hello1 crw-r--r--1rootroot234,0Jul142005/dev/hello1 Theparameterswepassedtomknodincludethename,type,andmajorandminornumbersforour devicedriver.Thenamewechose,ofcourse,washello1.Becausewearedemonstratingtheuseofa characterdriver,weusectoindicatethat.Themajornumberis234,thenumberwechoseforthis example,andtheminornumberis0. Byitself,thedevicenodeisjustanotherfileonourfilesystem.However,becauseofitsspecial status as a device node, we use it to bind to an installed device driver. If an application process issuesan open() system callwithourdevicenodeasthepathparameter,thekernelsearchesfor a valid device driver registered with a major number that matches the device nodein our case, 234. Thisisthemechanismbywhichthekernelassociatesourparticulardevicetothedevicenode. AsmostCprogrammersknow,theopen()systemcall,oranyofitsvariants,returnsareference (file descriptor) that our applications use to issue subsequent file system operations, such as read, write,and close.Thisreferenceisthenpassedtothevariousfilesystemoperations,suchasread, write,ortheirvariants. Forthosecuriousaboutthepurposeoftheminornumber,itisamechanismforhandlingmultiple devicesorsubdeviceswithasingledevicedriver.Itisnotusedbytheoperatingsystem;itissimply passedtothedevicedriver.Thedevicedrivercanusetheminornumberinanywayitseesfit.Asan example,withamultiportserialcard,themajornumberwouldspecifythedriver.Theminornumber mightspecifyoneofthemultipleportshandledbythesamedriveronthemultiportcard.Interested readersareencouragedtoconsultoneoftheexcellenttextsondevicedriversforfurtherdetails. 8.4.BringingItAllTogether Now that we have a skeletal device driver, we can load it and exercise it. Listing 8-11 is a simpleuserspaceapplicationthatexercisesourdevicedriver.We'vealreadyseenhowtoloadthe driver. Simply compile it and issue the make modules_install command to place it on your file system,aspreviouslydescribed. Listing8-11.ExercisingOurDeviceDriver #include<stdio.h> #include<stdlib.h> #include<sys/types.h> #include<sys/stat.h> #include<fcntl.h> #include<unistd.h> intmain(intargc,char**argv){ /*Ourfiledescriptor*/ intfd; intrc=0; char*rd_buf[16]; printf("%s:entered\n",argv[0]); /*Openthedevice*/ fd=open("/dev/hello1",O_RDWR); if(fd==-1){ perror("openfailed"); rc=fd; exit(-1); } printf("%s:open:successful\n",argv[0]); /*Issuearead*/ rc=read(fd,rd_buf,0); if(rc==-1){ perror("readfailed"); close(fd); exit(-1); } printf("%s:read:returning%dbytes!\n",argv[0],rc); close(fd); return0; } Thissimplefile,compiledonanARMXScalesystem,demonstratesthebindingofapplicationto devicedriver,throughthedevicenode.Likethedevicedriver,itdoesn'tdoanyusefulwork,butit doesdemonstratetheconceptsasitexercisessomeofthemethodsweintroducedinthedevicedriver ofListing8-10. Firstweissueanopen()systemcall[67]onourdevicenodecreatedearlier.Iftheopensucceeds, weindicatethatwithamessagetotheconsole.Nextweissuearead()commandandagainprinta messagetotheconsoleonsuccess.Noticethatareadof0bytesisperfectlyacceptableasfarasthe kernel is concerned and, in actual practice, indicates an end-of-file or out-of-data condition. Your device driver defines that special condition. When complete, we simply close the file and exit. Listing8-12capturestheoutputofrunningthisexampleapplicationonanARMXScaletarget: Listing8-12.UsingtheExampleDriver $modprobehello1 HelloExampleInit-debugmodeisdisabled Hello:registeredmodulesuccessfully! $./use-hello ./use-hello:entered ./use-hello:open:successful ./use-hello:read:returningzerobytes! $ 8.5.DeviceDriversandtheGPL MuchdiscussionanddebatesurroundstheissueofdevicedriversandhowthetermsoftheGNU PublicLicenseapplytodevicedrivers.Thefirsttestiswellunderstood:Ifyourdevicedriver(or anysoftware,forthatmatter)isbased,eveninpart,onexistingGPLsoftware,itiscalledaderived work.Forexample,ifyoustartwithacurrentLinuxdevicedriverandmodifyittosuityourneeds, this is certainly considered a derived work, and you are obligated to license this modified device driverunderthetermsoftheGPL,observingallitsrequirements. This is where the debate comes in. First, the disclaimer. This is not a legal opinion, and the authorisnotalawyer.Someoftheseconceptshavenotbeentestedincourtasofthiswriting.The prevailingopinionofthelegalandopensourcecommunitiesisthatifaworkcanbeproven[68]tobe independentlyderived,andagivendevicedriverdoesnotassume"intimateknowledge"oftheLinux kernel,thedevelopersarefreetolicenseitinanywaytheyseefit.Ifmodificationsaremadetothe kerneltoaccommodateaspecialneedofthedriver,itisconsideredaderivedworkand,therefore,is subjecttotheGPL. A large and growing body of information exists in the open source community regarding these issues.Itseemslikelythat,atsomepointinthefuture,theseconceptswillbetestedinacourtoflaw andprecedentwillbeestablished.Howlongthatmighttakeisanyone'sguess.Ifyouareinterestedin gainingabetterunderstandingofthelegalissuessurroundingLinuxandopensource,youmightenjoy www.open-bar.org. 8.6.ChapterSummary Thischapterpresentedahigh-leveloverviewofdevicedriverbasicsandhowtheyfitintothe architectureofaLinuxsystem.Armedwiththebasics,readersnewtodevicedriverscanjumpinto oneoftheexcellenttextsdevotedtodevicedriverwriters.ConsultSection8.6.1forreferences. • Device drivers enforce a rational separation between unprivileged user applications and critical kernel resources such as hardware and other devices, and present a well-known unified interfacetoapplications. •Theminimuminfrastructuretoloadadevicedriverisonlyafewlinesofcode.Wepresented thisminimuminfrastructureandbuiltontheconceptstoasimpleshellofadrivermodule. • Device drivers configured as loadable modules can be inserted into and removed from a runningkernelafterkernelboot. • Module utilities are used to manage the insertion, removal, and listing of device driver modules.Wecoveredthedetailsofthemoduleutilitiesusedforthesefunctions. •Devicenodesonyourfilesystemprovidethegluebetweenyouruserspaceapplicationandthe devicedriver. • Driver methods implement the familiar open, read, write, and close functionality commonly foundinUNIX/Linuxdevicedrivers.Thismechanismwasexplainedbyexample,includingasimple userapplicationtoexercisethesedrivermethods. • We concluded this chapter with an introduction to the relationship between kernel device driversandtheOpenSourceGNUPublicLicense. 8.6.1.SuggestionsforAdditionalReading LinuxDeviceDrivers,3rdEdition AlessandroRubiniandJonathanCorbet O'ReillyPublishing,2005 FilesystemHierarchyStandard EditedbyRustyRussel,DanielQuinlan,andChristopherYeoh TheFileSystemsHierarchyStandardsGroup www.pathname.com/fhs/ Rusty'sLinuxKernelPage ModuleUtilitiesfor2.6 RustyRussell http://kernel.org/pub/linux/kernel/people/rusty/ Chapter9.FileSystems Perhaps one of the most important decisions an embedded developer makes is which file system(s)todeploy.Somefilesystemsoptimizeforperformance,whereasothersoptimizeforsize. Still others optimize for data recovery after device or power failure. This chapter introduces the majorfilesystemsinuseonLinuxsystemsandexaminesthecharacteristicsofeachastheyapplyto embeddeddesigns.Itisnottheintentofthischaptertoexaminetheinternaltechnicaldetailsofeach file system. Instead, this chapter examines the operational characteristics and development issues related to each file system presented. References in Section 9.11.1, "Suggestions for Additional Reading,"areprovidedattheendofthechapterfortheinterestedreader. Starting with the most popular file system in use on earlier Linux desktop distributions, we introduceconceptsusingtheSecondExtendedFileSystem(ext2)tolaysomefoundationforfurther discussion.Nextwelookatitssuccessor,theThirdExtendedFileSystem(ext3),whichisthedefault filesystemformanypopulardesktopLinuxdistributionsbeingshippedtoday. Afterintroducingsomefundamentals,weexamineavarietyofspecializedfilesystems,including thoseoptimizedfordatarecoveryandforstoragespace,andthosedesignedforuseonFlashmemory devices. The Network File System (NFS) is presented, followed by a discussion of the more importantPseudoFileSystems,includingtheprocfilesystemandsysfs. 9.1.LinuxFileSystemConcepts Beforedelvingintothedetailsoftheindividualfilesystems,let'slookatthebigpictureofhow dataisstoredonaLinuxsystem.InourstudyofdevicedriversinChapter8,"DeviceDriverBasics," welookedatthestructureofacharacterdevice.Ingeneral,characterdevicesstoreandretrievedata in serial streams. The most basic example of a character device is a serial port or magnetic tape drive.Incontrast,blockdevicesstoreandretrievedatainequal-sizedchucksofdataatatime.For example, a typical IDE hard disk controller can transfer 512 bytes of data at a time to and from a specific,addressablelocationonthephysicalmedia.Filesystemsarebasedonblockdevices. 9.1.1.Partitions Before we begin our discussion of file systems, we start by introducing partitions, the logical divisionofaphysicaldeviceuponwhichafilesystemexists.Atthehighestlevel,dataisstoredon physical devices in partitions. A partition is a logical division of the physical medium (hard disk, Flash memory) whose data is organized following the specifications of a given partition type. A physicaldevicecanhaveasinglepartitioncoveringallitsavailablespace,oritcanbedividedinto multiplepartitionstosuitaparticulartask.Apartitioncanbethoughtofasalogicaldiskontowhich acompletefilesystemcanbewritten. Figure9-1showstherelationshipbetweenpartitionsandfilesystems. Figure9-1.Partitionsandfilesystems Linuxusesautilitycalledfdisktomanipulatepartitionsonblockdevices.Arecentfdiskutility found on many Linux distributions has knowledge of more than 90 different partition types. In practice, only a few are commonly used on Linux systems. Some common partition types include Linux,FAT32,andLinuxSwap. Listing9-1displaystheoutputofthefdiskutilitytargetingaCompactFlashdeviceconnectedtoa USBport.Onthisparticulartargetsystem,theUSBsubsystemassignedtheCompactFlashphysical devicetothedevicenode/dev/sdb. Listing9-1.DisplayingPartitionInformationUsingfdisk #fdisk/dev/sdb Command(mforhelp):p Disk/dev/sdb:49MB,49349120bytes 4heads,32sectors/track,753cylinders Units=cylindersof128*512=65536bytes DeviceBootStartEndBlocksIdSystem /dev/sdb1*11801150483Linux /dev/sdb21813601152083Linux /dev/sdb33615401152083Linux /dev/sdb45417531363283Linux Forthisdiscussion,wehavecreatedfourpartitionsonthedeviceusingthefdiskutility.Oneof themismarkedbootable,asindicatedbytheasteriskinthecolumnlabeledBoot.Thisissimplythe settingofaflaginthedatastructurethatrepresentsthepartitiontableonthedevice.Asyoucansee fromthelisting,thelogicalunitofstorageusedbyfdiskisacylinder.[69]Onthisdevice,acylinder contains64KB.Ontheotherhand,Linuxrepresentsthesmallestunitofstorageasalogicalblock. Youcandeducefromthislistingthatablockisaunitof1024bytes. AftertheCompactFlashhasbeenpartitionedinthismanner,eachdevicerepresentingapartition canbeformattedwithafilesystemofyourchoice.Whenapartitionisformattedwithagivenfile systemtype,Linuxcanmountthecorrespondingfilesystemfromthatpartition. 9.2.ext2 BuildingontheexampleofListing9-1,weneedtoformatthepartitionscreatedwithfdisk.Todo so, we use the Linux mke2fs utility. mke2fs is similar to the familiar DOS format command. This utilitymakesafilesystemoftypeext2onthespecifiedpartition.mke2fsisspecifictotheext2file system;otherfilesystemshavetheirownversionsoftheseutilities.Listing9-2capturestheoutputof thisprocess. Listing9-2.FormattingaPartitionUsingmke2fs #mke2fs/dev/sdb1-LCFlash_Boot_Vol mke2fs1.37(21-Mar-2005) Filesystemlabel=CFlash_Boot_Vol OStype:Linux Blocksize=1024(log=0) Fragmentsize=1024(log=0) 2880inodes,11504blocks 575blocks(5.00%)reservedforthesuperuser Firstdatablock=1 Maximumfilesystemblocks=11796480 2blockgroups 8192blockspergroup,8192fragmentspergroup 1440inodespergroup Superblockbackupsstoredonblocks: 8193 Writinginodetables:done Writingsuperblocksandfilesystemaccountinginformation:done Thisfilesystemwillbeautomaticallycheckedevery39mountsor 180days,whichevercomesfirst.Usetune2fs-cor-itooverride. # Listing 9-2 contains a great deal of detail relating to the ext2 file system and provides an excellent way to begin to understand its operational characteristics. Note that this partition was formattedastypeext2withavolumelabelofCFlash_Boot_Vol.ItwascreatedonaLinuxpartition (OSType:)withablocksizeof1024bytes.Spacewasallocatedfor2,880inodes,occupying11,504 blocks. An inode is the fundamental data structure representing a single file. For more detailed informationabouttheinternalstructureoftheext2filesystem,thereaderisdirectedtoSection9.11.1 attheendofthischapter. Lookingattheoutputofmke2fsinListing9-2,wecanascertaincertaincharacteristicsofhowthe storagedeviceisorganized.Wealreadyknowthattheblocksizeis1024bytes.Ifnecessaryforyour particular application, mke2fs can be instructed to format an ext2 file system with different block sizes.Currentimplementationsallowblocksizesof1,024,2,048,and4,096blocks. Blocksizeisalwaysacompromiseforbestperformance.Ononehand,largeblocksizeswaste morespaceondiskswithmanyfilesbecauseeachfilemustfitintoanintegralnumberofblocks.Any leftoverfragmentaboveblock_size*nmustoccupyanotherfullblock,evenifonly1byte.Onthe otherhand,verysmallblocksizesincreasethefilesystemoverheadofmanagingthemetadatathat describestheblock-to-filemapping.Benchmarktestingonyourparticularhardwareimplementation istheonlywaytobesureyouhaveselectedanoptimumblocksize. 9.2.1.MountingaFileSystem After a file system has been created, we can mount that file system on a running Linux system, provided that we have access to the hardware device and that the kernel has been compiled with supportforourparticularfilesystemtype,eitherasacompiled-inmoduleoradynamicallyloadable module.Thefollowingcommandmountsthepreviouslycreatedext2filesystemonamountpointthat wespecify: #mount/dev/sdb1/mnt/flash This example assumes that we have a directory created on our target Linux machine called /mnt/flash.Thisiscalledthemountpointbecauseweareinstalling(mounting)thefilesystemrooted atthispointinourfilesystemhierarchy.WearemountingtheFlashdevicedescribedearlierthatthe kernelassignedtothedevice/dev/sdb1.OnatypicalLinuxdesktop(development)machine,weneed tohaverootprivilegestoexecutethiscommand.[70]Themountpointisanyplaceonyourfilesystem thatyoudecide,whichbecomesthetoplevel(root)ofyournewlymounteddevice.Intheprevious example,toreferenceanyfilesonyourFlashdevice,youmustprefixthepathwith/mnt/flash. Themountcommandisapowerfulcommand,withmanyoptions.Manyoftheoptionsthatmount accepts depend on the target file system type of the mount operation. Most of the time, mount can determinethetypeoffilesystemonaproperlyformattedfilesystemknowntothekernel.Weprovide additionalusageexamplesforthemountcommandasweproceedthroughthischapter. Listing 9-3 displays the directory contents of a Flash device configured for an arbitrary embeddedsystem. Listing9-3.FlashDeviceListing $ls-l/mnt/flash total24 drwxr-xr-x2rootroot1024Jul1820:18bin drwxr-xr-x2rootroot1024Jul1820:18boot drwxr-xr-x2rootroot1024Jul1820:18dev drwxr-xr-x2rootroot1024Jul1820:18etc drwxr-xr-x2rootroot1024Jul1820:18home drwxr-xr-x2rootroot1024Jul1820:18lib drwx------2rootroot12288Jul1713:02lost+found drwxr-xr-x2rootroot1024Jul1820:18proc drwxr-xr-x2rootroot1024Jul1820:18root drwxr-xr-x2rootroot1024Jul1820:18sbin drwxr-xr-x2rootroot1024Jul1820:18tmp drwxr-xr-x2rootroot1024Jul1820:18usr drwxr-xr-x2rootroot1024Jul1820:18var $ Listing9-3isanexampleofwhatanembeddedsystemsrootfilesystemmightlooklikeatthetop (root) level. Chapter 6, "System Initialization," provides guidance and examples for how to determinethecontentsoftherootfilesystem. 9.2.2.CheckingFileSystemIntegrity The e2fsck command is used to check the integrity of an ext2 file system. A file system can becomecorruptedforseveralreasons,butbyfarthemostcommonreasonisanunexpectedpower failureorintentionalpower-downwithoutfirstclosingallopenfilesandunmountingthefilesystems. Linux distributions perform these operations during the shutdown sequence (assuming an orderly shutdownofthesystem).However,whenwearedealingwithembeddedsystems,unexpectedpowerdownsarecommon,andweneedtoprovidesomedefensivemeasuresagainstthesecases.e2fsckis ourfirstlineofdefenseforunexpectedpower-downusingtheext2filesystem. Listing9.4showstheoutputofe2fsckrunonourCompactFlashfromthepreviousexamples.It hasbeenformattedandproperlyunmounted;thereshouldbenoerrors. Listing9-4.CleanFileSystemCheck #e2fsck/dev/sdb1 e2fsck1.37(21-Mar-2005) CFlash_Boot_Vol:clean,23/2880files,483/11504blocks # Thee2fsckutilitychecksseveralaspectsofthefilesystemforconsistency.Ifnoissuesarefound, e2fsckissuesamessagesimilartothatshowninListing9-4.Notethate2fsckshouldberunonlyon an unmounted file system. Although it is possible to run it on a mounted file system, doing so can causesignificantdamagetointernalfilesystemstructuresonthediskorFlashdevice. To create a more interesting example, Listing 9-5 was created by pulling the CompactFlash deviceoutofitssocketwhilestillmounted.Weintentionallycreatedafileandeditingsessiononthat filebeforeremovingitfromthesystem.Thiscanresultincorruptionofthedatastructuresdescribing thefile,aswellastheactualdatablockscontainingthefile'sdata. Listing9-5.CorruptedFileSystemCheck #e2fsck-y/dev/sdb1 e2fsck1.37(21-Mar-2005) /dev/sdb1wasnotcleanlyunmounted,checkforced. Pass1:Checkinginodes,blocks,andsizes Inode13,i_blocksis16,shouldbe8.Fix?yes Pass2:Checkingdirectorystructure Pass3:Checkingdirectoryconnectivity Pass4:Checkingreferencecounts Pass5:Checkinggroupsummaryinformation /dev/sdb1:*****FILESYSTEMWASMODIFIED***** /dev/sdb1:25/2880files(4.0%non-contiguous),488/11504blocks # From Listing 9-5, you can see that e2fsck detected that the CompactFlash was not cleanly unmounted.Furthermore,youcanseetheprocessingonthefilesystemduringe2fsckchecking.The e2fsckutilitymakesfivepassesoverthefilesystem,checkingvariouselementsoftheinternalfile system'sdatastructures.Anerrorassociatedwithafile,identifiedbyinode[71]13,wasautomatically fixedbecausethe-yflagwasincludedonthee2fsckcommandline. Ofcourse,inarealsystem,youmightnotbethislucky.Sometypesoffilesystemerrorsarenot repairable using e2fsck. Moreover, the embedded system designer should understand that if power hasbeenremovedwithoutpropershutdown,thebootcyclecanbedelayedbythelengthoftimeit takestoscanyourbootdeviceandrepairanyerrors.Indeed,iftheseerrorsarenotrepairable,the system boot is halted and manual intervention is indicated. Furthermore, it should be noted that if your file system is large, the file system check (fsck) can take minutes or even hours for large multigigabytefilesystems. Another defense against file system corruption is to ensure that writes are committed to disk immediately when written. The sync utility can be used to force all queued I/O requests to be committedtotheirrespectivedevices.Onestrategytominimizethewindowofvulnerabilityfordata corruptionfromunexpectedpowerlossordrivefailureistoissuethesynccommandaftereveryfile writeorstrategicallyasneededbyyourapplicationrequirements.Thetrade-offhereis,ofcourse,a performance penalty. Deferring disk writes is a performance optimization used in all modern operatingsystems.Usingsynceffectivelydefeatsthisoptimization. The ext2 file system has matured as a fast, efficient, and robust file system for Linux systems. However, if you need the additional reliability of a journaling file system, or if boot time after uncleanshutdownisanissueinyourdesign,youshouldconsidertheext3filesystem. 9.3.ext3 Theext3filesystemhasbecomeapowerful,high-performance,androbustjournalingfilesystem. ItiscurrentlythedefaultfilesystemformanypopulardesktopLinuxdistributionssuchasRedHat andtheFedoraCoreseries. The ext3 file system is basically an extension of the ext2 file system with added journaling capability.Journalingisatechniqueinwhicheachchangetothefilesystemisloggedinaspecialfile sothatrecoveryispossiblefromknownjournalingpoints.Oneoftheprimaryadvantagesoftheext3 file system is its capability to be mounted directly after an unclean shutdown. As stated in the previoussection,whenasystemshutsdownunexpectedly,suchasduringapowerfailure,thesystem forcesafilesystemconsistencycheck,whichcanbealengthyoperation.Withext3filesystems,there is no need for a consistency check because the journal can simply be played back to ensure consistencyofthefilesystem. Without going into design details that are beyond the scope of this book, it is worth a quick explanationofhowajournalingfilesystemworks.Ajournalingfilesystemcontainsaspecialfile, often hidden from the user, that is used to store file system metadata[72] and file data itself. This specialfileisreferredtoasthejournal.Wheneverthefilesystemissubjecttoachange(suchasa writeoperation)thechangesarefirstwrittentothejournal.Thefilesystemdriversmakesurethat this write is committed to the journal before the actual changes are posted and committed to the storage media (disk or Flash, for example). After the changes have been logged in the journal, the driverpoststhechangestotheactualfileandmetadataonthemedia.Ifapowerfailureoccursduring themediawriteandarebootoccurs,allthatisnecessarytorestoreconsistencytothefilesystemis toreplaythechangesinthejournal. Oneofthemostsignificantdesigngoalsfortheext3filesystemwasthatitbebothbackwardand forwardcompatiblewiththeext2filesystem.Itispossibletoconvertanext2filesystemtoext3file systemandbackagainwithoutreformattingorrewritingallthedataonthedisk.Let'sseehowthisis done.[73]Listing9-6detailstheprocedure. Listing9-6.Convertingext2FileSystemtoext3FileSystem #mount/dev/sdb1/mnt/flash<<<Mounttheext2filesystem #tune2fs-j/dev/sdb1<<<Createthejournal tune2fs1.37(21-Mar-2005) Creatingjournalinode:done Thisfilesystemwillbeautomaticallycheckedevery23mountsor 180days,whichevercomesfirst.Usetune2fs-cor-itooverride. # Noticethatwefirstmountedthefilesystemon/mnt/flashforillustrativepurposesonly.Normally, we would execute this command on an unmounted ext2 partition. The design behavior for tune2fs whenthefilesystemismountedistocreatethejournalfilecalled.journal,ahiddenfile.Afilein Linuxprecededwiththeperiod(.)isconsideredahiddenfile;mostLinuxcommandlinefileutilities silentlyignorefilesofthistype.FromListing9-7,wecanseethatthelscommandwasinvokedwith the-aflag,whichtellsthelsutilitytolistallfiles. Listing9-7.ext3JournalFile $ls-al/mnt/flash total1063 drwxr-xr-x15rootroot1024Aug2519:25. drwxrwxrwx5rootroot4096Jul1819:49.. drwxr-xr-x2rootroot1024Aug1411:27bin drwxr-xr-x2rootroot1024Aug1411:27boot drwxr-xr-x2rootroot1024Aug1411:27dev drwxr-xr-x2rootroot1024Aug1411:27etc drwxr-xr-x2rootroot1024Aug1411:27home -rw-------1rootroot1048576Aug2519:25.journal drwxr-xr-x2rootroot1024Aug1411:27lib drwx------2rootroot12288Aug1411:27lost+found drwxr-xr-x2rootroot1024Aug1411:27proc drwxr-xr-x2rootroot1024Aug1411:27root drwxr-xr-x2rootroot1024Aug1411:27sbin drwxr-xr-x2rootroot1024Aug1411:27tmp drwxr-xr-x2rootroot1024Aug1411:27usr drwxr-xr-x2rootroot1024Aug1411:27var NowthatwehavecreatedthejournalfileonourFlashmodule,itiseffectivelyformattedasan ext3 file system. The next time the system is rebooted or the e2fsck utility is run on the partition containing the newly created ext3 file system, the journal file is automatically made invisible. Its metadataisstoredinareservedinodesetasideforthispurpose.Aslongasyoucanseethe.journal file,itisdangeroustomodifyordeletethisfile. It is possible and sometimes advantageous to create the journal file on a different device. For example, if you have more than one physical device on your system, you can place your ext3 journalingfilesystemonthefirstdriveandhavethejournalfileontheseconddrive.Thismethod worksregardlessofwhetheryourphysicalstorageisbasedonFlashorrotationalmedia.Tocreate thejournalingfilesystemfromanexistingext2filesystemwiththejournalfileinaseparatepartition, invoketune2fsinthefollowingmanner: #tune2fs-Jdevice=/dev/sda1-j/dev/sdb1 Forthistowork,youmusthavealreadyformattedthedevicewherethejournalistoresidewitha journalfileitmustbeanext3filesystem. 9.4.ReiserFS TheReiserFSfilesystemhasenjoyedpopularityamongsomedesktopdistributionssuchasSuSE andGentoo.Asofthiswriting,Reiser4isthecurrentincarnationofthisjournalingfilesystem.Like the ext3 file system, ReiserFS guarantees that either a given file system operation completes in its entiretyornoneofitcompletes.Unlikeext3,Reiser4hasintroducedanAPIforsystemprogrammers toguaranteetheatomicityofafilesystemtransaction.Considerthefollowingexample: Adatabaseprogramisbusyupdatingrecordsinthedatabase.Severalwritesareissuedtothefile system. Power is lost after the first write but before the last one has completed. A journaling file systemguaranteesthatthemetadatachangeshavebeenstoredtothejournalfilesothatwhenpower isagainappliedtothesystem,thekernelcanatleastestablishaconsistentstateofthefilesystem. That is, if file A was reported has having 16KB before the power failure, it will be reported as having16KBafterward,andthedirectoryentryrepresentingthisfile(actually,theinode)properly recordsthesizeofthefile.Thisdoesnotmean,however,thatthefiledatawasproperlywrittentothe file;itindicatesonlythattherearenoerrorsonthefilesystem.Indeed,itislikelythatdatawaslost bythedatabaseprograminthepreviousscenario,anditwouldbeuptothedatabaselogictorecover thelostdataifrecoveryistooccuratall. Reiser4 implements high-performance "atomic" file system operations designed to protect both thestateofthefilesystem(itsconsistency)andthedatainvolvedinafilesystemoperation.Reiser4 providesauser-levelAPItoenableprogramssuchasdatabasemanagerstoissueafilesystemwrite command that is guaranteed to either succeed in its entirety or fail in a similar manner, thus guaranteeingnotonlythatfilesystemconsistencyismaintained,butthatnopartialdataorgarbage dataremainsinfilesaftersystemcrash. FormoredetailsandtheactualsoftwareforReiserFS,visitthehomepagereferencedinSection 9.11.1attheendofthischapter. 9.5.JFFS2 Flashmemoryhasbeenusedextensivelyinembeddedproducts.BecauseofthenatureofFlash memorytechnology,itisinherentlylessefficientandmorepronetodatacorruptioncausedbypower lossfrommuchlargerwritetimes.Theinefficiencystemsfromtheblocksize.BlocksizesofFlash memorydevicesareoftenmeasuredinthetensorhundredsofkilobytes.Flashmemorycanbeerased onlyablockatatime,althoughwritescanusuallybeexecuted1byteorwordatatime.Toupdatea singlefile,anentireblockmustbeerasedandrewritten. It is well known that the distribution of file sizes on any given Linux machine (or other OS) contains many more smaller files than larger files. The histogram in Figure 9-2, generated with gnuplot,illustratesthedistributionoffilesizesonatypicalLinuxdevelopmentsystem. Figure9-2.Filesizesinbytes FromFigure9-2,wecanseethatthebulkofthefilesizesarewellbelowapproximately10KB. Thespikeat4096representsdirectories.Directoryentries(alsofilesthemselves)areexactly4096 bytes in length, and there are many of them. The spike above 40,000 bytes is an artifact of the measurement. It is a count of the number of files greater than approximately 40KB, the end of the measurementquantum.Itisinterestingtonotethatthevastmajorityoffilesareverysmall. Small file sizes present a unique challenge to the Flash file system designer. Because Flash memory must be erased one entire block at a time, and the size of a Flash block is often many multiplesofthesmallerfilesizes,Flashissubjecttotime-consumingblockrewriting.Forexample, assumethata128KBblockofFlashisbeingusedtoholdacoupledozenfilesof4096bytesorless. Now assume that one of those files needs to be modified. This causes the Flash file system to invalidatetheentire128KBblockandrewriteeveryfileintheblocktoanewlyerasedblock.This canbeatime-consumingprocess. BecauseFlashwritescanbetime-consuming(muchslowerthanharddiskwrites),thisincreases thewindowwheredatacorruptioncanoccurduetosuddenlossofpower.Unexpectedpowerlossis acommonoccurrenceinembeddedsystems.Forinstance,ifpowerislostduringtherewriteofthe 128KB data block referenced in the previous paragraph, all of the couple dozen files could potentiallybelost. Enter JFFS2. These issues just discussed and other problems have been largely reduced or eliminated by the design of the second-generation Journaling Flash File System, or JFFS2. The originalJFFSwasdesignedbyAxisCommunicationsABofSwedenandwastargetedspecificallyat the commonly available Flash memory devices at the time. The JFFS had knowledge of the Flash architectureand,moreimportant,architecturallimitationsimposedbythedevices. Another problem with Flash file systems is that Flash memory has a limited lifetime. Typical Flash memory devices are specified for a minimum of 100,000 write cycles, and, more recently, 1,000,000-cycledeviceshavebecomecommon.Thisspecificationisapplicabletoeachblockofthe Flashdevice.Thisunusuallimitationimposestherequirementtospreadthewritesevenlyacrossthe blocksofaFlashmemorydevice.JFFS2usesatechniquecalledwearlevelingtoaccomplishthis function. Building a JFFS2 image is relatively straightforward. As always, you must ensure that your kernelhassupportforJFFS2andthatyourdevelopmentworkstationcontainsacompatibleversionof themkfs.jffs2utility.JFFS2imagesarebuiltfromadirectorythatcontainsthedesiredfilesonthefile systemimage.Listing9-8showsatypicaldirectorystructureforaFlashdevicedesignedtobeused asarootfilesystem. Listing9-8.DirectoryLayoutforJFFS2FileSystem $ls-l total44 drwxr-xr-x2rootroot4096Aug1411:27bin drwxr-xr-x2rootroot4096Aug1411:27dev drwxr-xr-x2rootroot4096Aug1411:27etc drwxr-xr-x2rootroot4096Aug1411:27home drwxr-xr-x2rootroot4096Aug1411:27lib drwxr-xr-x2rootroot4096Aug1411:27proc drwxr-xr-x2rootroot4096Aug1411:27root drwxr-xr-x2rootroot4096Aug1411:27sbin drwxr-xr-x2rootroot4096Aug1411:27tmp drwxr-xr-x2rootroot4096Aug1411:27usr drwxr-xr-x2rootroot4096Aug1411:27var $ Whensuitablypopulatedwithruntimefiles,thisdirectorylayoutcanbeusedasatemplateforthe mkfs.jffs2 command. The mkfs.jffs2 command produces a properly formatted JFFS2 file system imagefromadirectorytreesuchasthatinListing9-8.Commandlineparametersareusedtopass mkfs.jffs2thedirectorylocationaswellasthenameoftheoutputfiletoreceivetheJFFS2image. ThedefaultistocreatetheJFFS2imagefromthecurrentdirectory.Listing9-9showsthecommand forbuildingtheJFFS2image. Listing9-9.mkfs.jffs2CommandExample #mkfs.jffs2-d./jffs2-image-dir-ojffs2.bin #ls-l total4772 -rw-r--r--1rootroot1098640Sep1722:03jffs2.bin drwxr-xr-x13rootroot4096Sep1722:02jffs2-image-dir # The directory structure and files from Listing 9-8 are in the jffs2-image-dir directory in our example. We arbitrarily execute the mkfs.jffs2 command from the directory above our file system image.Usingthe-dflag,wetellthemkfs.jffs2commandwherethefilesystemtemplateislocated. Weusethe-oflagtonametheoutputfiletowhichtheresultingJFFS2imageiswritten.Theresulting image,jffs2.bin,isusedinChapter10,"MTDSubsystem,"whenweexaminetheJFFS2filetogether withtheMTDsubsystem. ItshouldbepointedoutthatanyFlash-basedfilesystemthatsupportswriteoperationsissubject toconditionsthatcanleadtoprematurefailureoftheunderlyingFlashdevice.Forexample,enabling system loggers (syslogd and klogd) configured to write their data to Flash-based file systems can easilyoverwhelmaFlashdevicewithcontinuouswrites.Somecategoriesofprogramerrorscanalso leadtocontinuouswrites.CaremustbetakentolimitFlashwritestovalueswithinthelifetimeof Flashdevices. 9.6.cramfs FromtheREADMEfileinthecramfsproject,thegoalofcramfsisto"cramafilesystemintoa smallROM."ThecramfsfilesystemisveryusefulforembeddedsystemsthatcontainasmallROM orFLASHmemorythatholdsstaticdataandprograms.BorrowingagainfromthecramfsREADME file,"cramfsisdesignedtobesimpleandsmall,andcompressthingswell." Thecramfsfilesystemisreadonly.Itiscreatedwithacommandlineutilitycalledmkcramfs.If youdon'thaveitonyourdevelopmentworkstation,youcandownloaditfromthelinkattheendof thischapter.AswithJFFS2,mkcramfsbuildsafilesystemimagefromadirectoryspecifiedonthe commandline.Listing9-10detailstheprocedureforbuildingacramfsimage.Weusethesamefile systemstructurefromListing9-8thatweusedtobuildtheJFFS2image. Listing9-10.mkcramfsCommandExample #mkcramfs usage:mkcramfs[-h][-v][-bblksize][-eedition][-ifile][-nname] dirnameoutfile -hprintthishelp -Emakeallwarningserrors(non-zeroexitstatus) -bblksizeusethisblocksize,mustequalpagesize -eeditionseteditionnumber(partoffsid) -ifileinsertafileimageintothefilesystem(requires>=2.4.0) -nnamesetnameofcramfsfilesystem -ppadby512bytesforbootcode -ssortdirectoryentries(oldoption,ignored) -vbemoreverbose -zmakeexplicitholes(requires>=2.3.39) dirnamerootofthedirectorytreetobecompressed outfileoutputfile # #mkcramfs.../cramfs.image warning:gidstruncatedto8bits(thismaybeasecurityconcern) #ls-l../cramfs.image -rw-rw-r--1chrischris1019904Sep1918:06../cramfs.image Themkcramfscommandwasinitiallyissuedwithoutanycommandlineparameterstoreproduce theusagemessage.Becausethereisnomanpageforthisutility,thisisthebestwaytounderstandits usage.Wesubsequentlyissuedthecommandspecifyingthecurrentdirectory,.,asthesourceofthe filesforthecramfsfilesystem,andafilecalledcramfs.imageasthedestination.Finally,welisted thefilejustcreated,andweseeanewfilecalledcramfs.image. Notethatifyourkernelisconfiguredwithcramfssupport,youcanmountthisfilesystemimage onyourLinuxdevelopmentworkstationandexamineitscontents.Ofcourse,becauseitisaread-only file system, you cannot modify it. Listing 9-11 demonstrates mounting the cramfs file system on a mountpointcalled/mnt/flash. Listing9-11.ExaminingthecramfsFileSystem #mount-oloopcramfs.image/mnt/flash #ls-l/mnt/flash total6 drwxr-xr-x1rootroot704Dec311969bin drwxr-xr-x1rootroot0Dec311969dev drwxr-xr-x1rootroot416Dec311969etc drwxr-xr-x1rootroot0Dec311969home drwxr-xr-x1rootroot172Dec311969lib drwxr-xr-x1rootroot0Dec311969proc drws------1rootroot0Dec311969root drwxr-xr-x1rootroot272Dec311969sbin drwxrwxrwt1rootroot0Dec311969tmp drwxr-xr-x1rootroot124Dec311969usr drwxr-xr-x1rootroot212Dec311969var # You might have noticed the warning message regarding group ID (GID) when the mkcramfs commandwasexecuted.Thecramfsfilesystemusesverytersemetadatatoreducefilesystemsize andincreasethespeedofexecution.Oneofthe"features"ofthecramfsfilesystemisthatittruncates the group ID field to 8 bits. Linux uses 16-bit group ID field. The result is that files created with groupIDsgreaterthan255aretruncatedwiththewarningissuedinListing9-10. Althoughsomewhatlimitedintermsofmaximumfilesizes,maximumnumberoffiles,andsoon, thecramfsfilesystemisidealforbootROMS,inwhichread-onlyoperationandfastcompression areideallysuited. 9.7.NetworkFileSystem ThoseofyouwhohavedevelopedintheUNIXenvironmentwillundoubtedlybefamiliarwith theNetworkFileSystem,orsimplyNFS.Properlyconfigured,NFSenablesyoutoexportadirectory on an NFS server and mount that directory on a remote client machine as if it were a local file system.ThisisusefulingeneralforlargenetworksofUNIX/Linuxmachines,anditcanbeapanacea to the embedded developer. Using NFS on your target board, an embedded developer can have access to a huge number of files, libraries, tools, and utilities during development and debugging, evenifthetargetembeddedsystemisresourceconstrained. As with the other file systems, your kernel must be configured with NFS support, for both the server-side functionality and the client side. NFS server and client functionality is independently configuredinthekernelconfiguration. Detailed instructions for configuring and tuning NFS are beyond the scope of this book, but a short introduction helps to illustrate how useful NFS can be in the embedded environment. See Section9.11.1attheendofthischapterforapointertodetailedinformationaboutNFS,includingthe completeNFS-Howto. OnyourdevelopmentworkstationwithNFSenabled,afilecontainsthenamesofeachdirectory thatyouwanttoexportviatheNetworkFileSystem.OnRedHatandotherdistributions,thisfileis locatedinthe/etcdirectoryandisnamedexports.Listing9-12illustratesasample/etc/exportssuch asmightbefoundonadevelopmentworkstationusedforembeddeddevelopment. Listing9-12.Contentsof/etc/exports $cat/etc/exports #/etc/exports /home/chris/sandbox/coyote-target*(rw,sync,no_root_squash) /home/chris/workspace*(rw,sync,no_root_squash) $ This file contains the names of two directories on a Linux development workstation. The first directorycontainsatargetfilesystemforanADIEngineeringCoyotereferenceboard.Thesecond directory is a general workspace that contains projects targeted for an embedded system. This is arbitrary;youcansetthingsupanywayyouchoose. On an embedded system with NFS enabled, the following command mounts the .../workspace directoryexportedbytheNFSserveronamountpointofourchoosing: $mount-tnfspluto:/home/chris/workspace/workspace Notice some important points about this command. We are instructing the mount command to mountaremotedirectory(onamachinenamedpluto)ontoalocalmountpointcalled/workspace. Forthissyntaxtowork,tworequirementsmustbemetontheembeddedtarget.First,forthetargetto recognizethesymbolicmachinenamepluto,itmustbecapableofresolvingthesymbolicname.The easiest way to do this is to place an entry in the /etc/hosts file on the target. This allows the networkingsubsystemtoresolvethesymbolicnametoitscorrespondingIPaddress.Theentryinthe target's/etc/hostsfilewouldlooklikethis: 192.168.10.9pluto The second requirement is that the embedded target must have a directory in its root directory called/workspace.Thisiscalledamountpoint.Thepreviousmountcommandcausesthecontentsof the NFS server's /home/chris/workspace directory to be available on the embedded system's /workspacepath. This is quite useful, especially in a cross-development environment. Let's say that you are workingonalargeprojectforyourembeddeddevice.Eachtimeyoumakechangestotheproject, you need to move that application to your target so you can test and debug it. Using NFS in the mannerjustdescribed,assumingthatyouareworkingintheNFSexporteddirectoryonyourhost,the changes are immediately available on your target embedded system without the need to upload the newlycompiledprojectfiles.Thiscanspeeddevelopmentconsiderably. 9.7.1.RootFileSystemonNFS Mountingyourprojectworkspaceonyourtargetembeddedsystemisveryusefulfordevelopment and debugging because it facilitates rapid access to changes and source code for source-level debugging. This is especially useful when the target system is severely resource constrained. NFS reallyshinesasadevelopmenttoolwhenyoumountyourembeddedsystem'srootfilesystementirely from an NFS server. From Listing 9-12, notice the coyote-target enTRy. This directory on your development workstation could contain hundreds or thousands of files compatible with your target architecture. TheleadingembeddedLinuxdistributionstargetedatembeddedsystemsshiptensofthousandsof filescompiledandtestedforthechosentargetarchitecture.Toillustratethis,Listing9-13containsa directorylistingofthecoyote-targetdirectoryreferencedinListing9-12. Listing9-13.TargetFileSystemExampleSummary $du-h--max-depth=1 724M./usr 4.0K./opt 39M./lib 12K./dev 27M./var 4.0K./tmp 3.6M./boot 4.0K./workspace 1.8M./etc 4.0K./home 4.0K./mnt 8.0K./root 29M./bin 32M./sbin 4.0K./proc 64K./share 855M. $ $find-typef|wc-l 29430 ThistargetfilesystemcontainsjustshyofagigabyteworthofbinaryfilestargetedattheARM architecture. As you can see from the listing, this is more than 29,000 binary, configuration and documentation files. This would hardly fit on the average Flash device found on an embedded system! This is the power of an NFS root mount. For development purposes, it can only increase productivityifyourembeddedsystemisloadedwithallthetoolsandutilitiesyouarefamiliarwith onaLinuxworkstation.Indeed,likelydozensofcommandlinetoolsanddevelopmentutilitiesthat youhave neverseen canhelp youshavetimeoffyourdevelopmentschedule.Youwilllearnmore aboutsomeoftheseusefultoolsinChapter13,"DevelopmentTools." ToenableyourembeddedsystemtomountitsrootfilesystemviaNFSatboottimeisrelatively straightforward. First, you must configure your target's kernel for NFS support. There is also a configurationoptiontoenablerootmountingofanNFSremotedirectory.ThisisillustratedinFigure 9-3. Figure9-3.NFSkernelconfiguration Notice that the NFS file system support has been selected, along with support for "Root file systemonNFS."Afterthesekernel-configurationparametershavebeenselected,allthatremainsis tosomehowfeedinformationtothekernelsothatitknowswheretolookfortheNFSserver.Several methods can be used for this, and some depend on the chosen target architecture and choice of bootloader.Ataminimum,thekernelcanbepassedtheproperparametersonthekernelcommand linetoconfigureitsIPportandserverinformationonpower-up.Atypicalkernelcommandlinemight looklikethis: console=ttyS0,115200ip=bootproot=/dev/nfs ThistellsthekerneltoexpectarootfilesystemviaNFSandtoobtaintherelevantparameters (server name, server IP address, and root directory to mount) from a BOOTP server. This is a commonandtremendouslyusefulconfigurationduringthedevelopmentphaseofaproject.Ifyouare staticallyconfiguringyourtarget'sIPaddress,yourkernelcommandlinemightlooklikethis: console=ttyS0,115200\ ip=192.168.1.139:192.168.1.1:192.168.1.1:255.255.255.0:coyote1:eth0:off\ nfsroot=192.168.1.1:/home/chris/sandbox/coyote-target\ root=/dev/nfs Ofcourse,thiswouldallbeononeline.Theip=parameterisdefinedin.../net/ipv4/ipconfig.c andhasthefollowingsyntax,allononeline: ip=<client-ip>:<server-ip>:<gw-ip>:<netmask>:<hostname>:<device>:<PROTO> Here,client-ipisthetarget'sIPaddress;server-ipistheaddressoftheNFSserver;gw-ipisthe gateway(router),incasetheserver-ipisonadifferentsubnet;andnetmaskdefinestheclassofIP addressing. hostname is a string that is passed as the target hostname; device is the Linux device name,suchaseth0;andPROTOdefinestheprotocolusedtoobtaininitialIPparameters. 9.8.PseudoFileSystems A number of file systems fall under the category of Pseudo File Systems in the kernelconfigurationmenu.Togethertheyprovidearangeoffacilitiesusefulinawiderangeofapplications. Foradditionalinformation,especiallyontheprocfilesystem,spendanafternoonpokingaroundthis usefulsystemfacility.Whereappropriate,referencestoadditionalreadingmaterialcanbefoundin Section9.11.1,attheendofthischapter. 9.8.1.ProcFileSystem The/procfilesystemtookitsnamefromitsoriginalpurpose,aninterfacethatallowsthekernel tocommunicateinformationabouteachrunningprocessonaLinuxsystem.Overthecourseoftime,it hasgrownandmaturedtoprovidemuchmorethanprocessinformation.Weintroducethehighlights here;acompletetourofthe/procfilesystemisleftasanexerciseforthereader. The /proc file system has become a virtual necessity for all but the simplest of Linux systems, even embedded ones. Many user-level functions rely on the contents of the /proc file system to do their job. For example, the mount command, issued without any parameters, lists all the currently activemountpointsonarunningsystem,fromtheinformationdeliveredby/proc/mounts.Ifthe/proc filesystemisnotavailable,themountcommandsilentlyreturns.Listing9-14illustratesthisonthe ADIEngineeringCoyoteboard. Listing9-14.MountDependencyon/proc #mount rootfson/typerootfs(rw) /dev/rooton/typenfs (rw,v2,rsize=4096,wsize=4096,hard,udp,nolock,addr=192.168.1.19) tmpfson/dev/shmtypetmpfs(rw) /procon/proctypeproc(rw,nodiratime) <Nowunmountprocandtryagain...> #umount/proc #mount # NoticeinListing9-14that/procitselfislistedasamountedfilesystem,astypeprocmountedon /proc. This is not doublespeak; your system must have a mount point called /proc at the top-level directorytreeasadestinationforthe/procfilesystemtobemountedon.[74]Tomountthe/procfile system,usethemountcommandaswithanyotherfilesystem: $mount-tproc/proc/proc The general form of the mount command, from the man page, is mount [-t fstype] something somewhere.Inthepreviousinvocation,wecouldhavesubstitutednonefor/proc,asfollows: $mount-tprocnone/proc This looks somewhat less like doublespeak. The something parameter is not strictly necessary because/procisapseudofilesystemandnotarealphysicaldevice.However,specifying/procasin theearlierexamplehelpsremindusthatwearemountingthe/procfilesystemonthe/procdirectory (or,moreappropriately,onthe/procmountpoint). Ofcourse,bythistime,itmightbeobviousthattoget/procfilesystemfunctionality,itmustbe enabledinthekernelconfiguration.Thiskernel-configurationoptioncanbefoundintheFileSystems submenuunderthecategoryPseudoFileSystems. Eachuserprocessrunninginthekernelisrepresentedbyanentryinthe/procfilesystem.For example, the init process introduced in Chapter 6 is always assigned the process id (PID) of 1. Processesinthe/procfilesystemarerepresentedbyadirectorythatisgiventhePIDnumberasits name. For example, the init process with a PID of 1 would be represented by a /proc/1 directory. Listing9-15showsthecontentsofthisdirectoryonourembeddedCoyoteboard. Listing9-15.initProcess/procEnTRies #ls-l/proc/1 total0 -r--------1rootroot0Jan100:25auxv -r--r--r--1rootroot0Jan100:21cmdline lrwxrwxrwx1rootroot0Jan100:25cwd->/ -r--------1rootroot0Jan100:25environ lrwxrwxrwx1rootroot0Jan100:25exe->/sbin/init dr-x------2rootroot0Jan100:25fd -r--r--r--1rootroot0Jan100:25maps -rw-------1rootroot0Jan100:25mem -r--r--r--1rootroot0Jan100:25mounts -rw-r--r--1rootroot0Jan100:25oom_adj -r--r--r--1rootroot0Jan100:25oom_score lrwxrwxrwx1rootroot0Jan100:25root->/ -r--r--r--1rootroot0Jan100:21stat -r--r--r--1rootroot0Jan100:25statm -r--r--r--1rootroot0Jan100:21status dr-xr-xr-x3rootroot0Jan100:25task -r--r--r--1rootroot0Jan100:25wchan Theseentries,whicharepresentinthe/procfilesystemforeachrunningprocess,containmuch usefulinformation,especiallyforanalyzinganddebuggingaprocess.Forexample,thecmdlineentry containsthecompletecommandlineusedtoinvoketheprocess,includinganyarguments.Thecwd androotdirectoriescontaintheprocesses'viewofthecurrentworkingdirectoryandthecurrentroot directory. Oneofthemoreusefulentriesforsystemdebuggingisthemapsentry.Thiscontainsalistofeach virtualmemorysegmentassignedtotheprogram,alongwithattributesabouteach.Listing9-16isthe outputfrom/proc/1/mapsinourexampleoftheinitprocess. Listing9-16.initProcessMemorySegmentsfrom/proc #cat/proc/1/maps 00008000-0000f000r-xp0000000000:0a9537567/sbin/init 00016000-00017000rw-p0000600000:0a9537567/sbin/init 00017000-0001b000rwxp0001700000:000 40000000-40017000r-xp0000000000:0a9537183/lib/ld-2.3.2.so 40017000-40018000rw-p4001700000:000 4001f000-40020000rw-p0001700000:0a9537183/lib/ld-2.3.2.so 40020000-40141000r-xp0000000000:0a9537518/lib/libc-2.3.2.so 40141000-40148000---p0012100000:0a9537518/lib/libc-2.3.2.so 40148000-4014d000rw-p0012000000:0a9537518/lib/libc-2.3.2.so 4014d000-4014f000rw-p4014d00000:000 befeb000-bf000000rwxpbefeb00000:000 # Theusefulnessofthisinformationisreadilyapparent.Youcanseetheprogramsegmentsofthe initprocessitselfinthefirsttwoentries.Youcanalsoseethememorysegmentsusedbytheshared libraryobjectsbeingusedbytheinitprocess.Theformatisasfollows: vmstart-vmendattrpgoffsetdevnameinodefilename Here,vmstartandvmendarethestartingandendingvirtualmemoryaddresses,respectively;attr indicatesmemoryregionattributes,suchasread,write,andexecute,andtellswhetherthisregionis shareable;pgoffsetisthepageoffsetoftheregion(akernelvirtualmemoryparameter);anddevname, displayedasxx:xx,isakernelrepresentationofthedeviceIDassociatedwiththismemoryregion. Thememoryregionsthatarenotassociatedwithafilearealsonotassociatedwithadevice,thusthe 00:00. The final two entries are the inode and file associated with the given memory region. Of course,ifthereisnofile,thereisnoinodeassociatedwithit,anditdisplayswithazero.Theseare usuallydatasegments. Otherusefulentriesarelistedforeachprocess.Thestatusentrycontainsusefulstatusinformation abouttherunningprocess,includingitemssuchastheparentPID,userandgroupIDs,virtualmemory usagestats,signals,andcapabilities.Moredetailscanbeobtainedfromthereferencesattheendof thechapter. Somefrequentlyused/procenTRiesarecpuinfo,meminfo,andversion.ThecpuinfoenTRylists attributesthatthekerneldiscoversabouttheprocessor(s)runningonthesystem.ThememinfoenTRy provides statistics on the total system memory. The version entry mirrors the Linux kernel version string,togetherwithinformationonwhatcompilerandmachinewereusedtobuildthekernel. Manymoreuseful/procentriesareprovidedbythekernel;wehaveonlyscratchedthesurfaceof this useful subsystem. Many utilities have been designed for extracting and reporting information containedwiththe/procfilesystem.Twopopularexamplesaretopandps,whicheveryembedded Linux developer should be intimately familiar with. These are introduced in Chapter 13. Other utilitiesusefulforinterfacingwiththe/procfilesystemincludefree,pkill,pmap,anduptime.Seethe procpspackageformoredetails. 9.8.2.sysfs Likethe/procfilesystem,sysfsisnotrepresentativeofanactualphysicaldevice.Instead,sysfs models specific kernel objects such as physical devices and provides a way to associate devices withdevicedrivers.SomeagentsinatypicalLinuxdistributiondependontheinformationonsysfs. Wecangetsomeideaofwhatkindsofobjectsareexportedbylookingdirectlyatthedirectory structureexportedbysysfs.Listing9-17showsthetop-level/sysdirectoryonourCoyoteboard. Listing9-17.Top-Level/sysDirectoryContents #dir/sys total0 drwxr-xr-x21rootroot0Jan100:00block drwxr-xr-x6rootroot0Jan100:00bus drwxr-xr-x10rootroot0Jan100:00class drwxr-xr-x5rootroot0Jan100:00devices drwxr-xr-x2rootroot0Jan100:00firmware drwxr-xr-x2rootroot0Jan100:00kernel drwxr-xr-x5rootroot0Jan100:00module drwxr-xr-x2rootroot0Jan100:00power # Asyoucansee,sysfsprovidesasubdirectoryforeachmajorclassofsystemdevice,including thesystembuses.Forexample,undertheblocksubdirectory,eachblockdeviceisrepresentedbya subdirectoryentry.Thesameholdstruefortheotherdirectoriesatthetoplevel. Mostoftheinformationstoredbysysfsisinaformatmoresuitableformachinesthanhumansto read.Forexample,todiscoverthedevicesonthePCIbus,onecouldlookdirectlyatthe/sys/bus/pci subdirectory.OnourCoyoteboard,whichhasasinglePCIdeviceattached(anEthernetcard),the directorylookslikethis: #ls/sys/bus/pci/devices/ 0000:00:0f.0->../../../devices/pci0000:00/0000:00:0f.0 Thisentryisactuallyasymboliclinkpointingtoanothernodeinthesysfsdirectorytree.Wehave formatted the output of ls here to illustrate this, while still fitting in a single line. The name of the symbolic link is the kernel's representation of the PCI bus, and it points to a devices subdirectory calledpci0000:00(thePCIbusrepresentation),whichcontainsanumberofsubdirectoriesandfiles representing attributes of this specific PCI device. As you can see, the data is rather difficult to discoverandparse. Ausefulutilityexiststobrowsethesysfsfilesystemdirectorystructure.Calledsystool,itcomes fromthesysfsutilspackagefoundonsourceforge.net.HereishowsystoolwoulddisplaythePCIbus fromthepreviousdiscussion: $systool-bpci Bus="pci" 0000:00:0f.08086:1229 Againweseethekernel'srepresentationofthebusanddevice(0f),butthistimethetooldisplays the vendor ID (8086 = Intel) and device ID (1229 = eepro100 Ethernet card) obtained from the /sys/devices/pci0000:00branchof/syswheretheseattributesarekept.Executedwithnoparameters, systooldisplaysthetop-levelsystemhierarchy.Listing9-18isanexamplefromourCoyoteboard. Listing9-18.Outputfromsystool $systool Supportedsysfsbuses: i2c ide pci platform Supportedsysfsclasses: block i2c-adapter i2c-dev input mem misc net pci_bus tty Supportedsysfsdevices: pci0000:00 platform system Youcanseefromthislistingthevarietyofsysteminformationavailablefromsysfs.Manyutilities usethisinformationtodeterminethecharacteristicsofsystemdevicesortoenforcesystempolicies, suchaspowermanagementandhot-plugcapability. 9.9.OtherFileSystems NumerousfilesystemsaresupportedunderLinux.Spacedoesnotpermitcoverageofallofthem. However,youshouldbeawareofsomeotherimportantfilesystemsfrequentlyfoundinembedded systems. TheramfsfilesystemisbestconsideredfromthecontextoftheLinuxsourcecodemodulethat implementsit.Listing9-19reproducesthefirstseverallinesofthatfile. Listing9-19.LinuxramfsSourceModuleComments /* *ResizablesimpleramfilesystemforLinux. * *Copyright(C)2000LinusTorvalds. *2000TransmetaCorp. * *UsagelimitsaddedbyDavidGibson,LinuxcareAustralia. *ThisfileisreleasedundertheGPL. */ /* *NOTE!Thisfilesystemisprobablymostuseful *notasarealfilesystem,butasanexampleof *howvirtualfilesystemscanbewritten. * *Itdoesn'tgetmuchsimplerthanthis.Consider *thatthisfileimplementsthefullsemanticsof *aPOSIX-compliantread-writefilesystem. Thismodulewaswrittenprimarilyasanexampleofhowvirtualfilesystemscanbewritten.One oftheprimarydifferencesbetweenthisfilesystemandtheramdiskfacilityfoundin modernLinux kernels is its capability to shrink and grow according to its use. A ramdisk does not have this property. This source module is compact and well written. It is presented here for its educational value.Youareencouragedtostudythisgoodexample. Thetmpfsfilesystemissimilartoandrelatedtorams.Likeramfs,everythingintmpfsisstored inkernelvirtualmemory,andthecontentsoftmpfsarelostonpower-downorreboot.Thetmpfsfile system is useful for fast temporary storage of files. I use tmpfs mounted on /tmp in a midi/audio application to speed up the creation and deletion of temporary objects required by the audio subsystem.Thisisalsoagreatwaytokeepyour/tmpdirectorycleanitscontentsarelostonevery reboot.Mountingtmpfsissimilartoanyothervirtualfilesystem: #mount-ttmpfs/tmpfs/tmp Aswithothervirtualfilesystemssuchas/proc,thefirst/tmpfsparameterinthepreviousmount command is a "no-op"that is, it could be the word none and still function. However, it is a good reminderthatyouaremountingavirtualfilesystemcalledtmpfs. 9.10.BuildingaSimpleFileSystem It is straightforward to build a simple file system image. Here we demonstrate the use of the Linux kernel's loopback device. The loopback device enables the use of a regular file as a block device.Inshort,webuildafilesystemimageinaregularfileandusetheLinuxloopbackdeviceto mountthatfileinthesamewayanyotherblockdeviceismounted. Tobuildasimplerootfilesystem,startwithafixed-sizedfilecontainingallzeros: #ddif=/dev/zeroof=./my-new-fs-imagebs=1kcount=512 Thiscommandcreatesafileof512KBcontainingnothingbutzeros.Wefillthefilewithzerosto aidincompressionlaterandtohaveaconsistentdatapatternforuninitializeddatablockswithinthe filesystem.Usecautionwiththeddcommand.Executingddwithnoboundary(count=)orwithan improperboundarycanfillupyourharddriveandpossiblycrashyoursystem.ddisapowerfultool; use it with the respect it deserves. Simple typos in commands such as dd, executed as root, have destroyedcountlessfilesystems. When we have the new image file, we actually format the file to contain the data structures definedbyagivenfilesystem.Inthisexample,webuildanext2filesystem.Listing9-20detailsthe procedure. Listing9-20.Creatinganext2FileSystemImage #/sbin/mke2fs./my-new-fs-image mke2fs1.35(28-Feb-2004) ./my-new-fs-imageisnotablockspecialdevice. Proceedanyway?(y,n)y Filesystemlabel= OStype:Linux Blocksize=1024(log=0) Fragmentsize=1024(log=0) 64inodes,512blocks 25blocks(4.88%)reservedforthesuperuser Firstdatablock=1 1blockgroup 8192blockspergroup,8192fragmentspergroup 64inodespergroup Writinginodetables:done Writingsuperblocksandfilesystemaccountinginformation:done Thisfilesystemwillbeautomaticallycheckedevery24mountsor 180days,whichevercomesfirst.Usetune2fs-cor-itooverride. # Aswithdd,themke2fscommandcandestroyyoursystem,souseitwithcare.Inthisexample, weaskedmke2fstoformatafileratherthanaharddrivepartition(blockdevice)forwhichitwas intended.Assuch,mke2fsdetectedthatfactandaskedustoconfirmtheoperation.Afterconfirming, mke2fsproceededtowriteanext2superblockandfilesystemdatastructuresintothefile.Wethen canmountthisfilelikeanyblockdevice,usingtheLinuxloopbackdevice: #mount-oloop./my-new-fs-image/mnt/flash This command mounts the file my-new-fs-image as a file system on the mount point named /mnt/flash.Themountpointnameisnotimportant;youcanmountitwhereveryouwant,aslongasthe mountpointexists.Usemkdirtocreateyourmountpoint. Afterthenewlycreatedimagefileismountedasafilesystem,wearefreetomakechangestoit. Wecanaddanddeletedirectories,makedevicenodes,andsoon.Wecanusetartocopyfilesintoor outofit.Whenthechangesarecomplete,theyaresavedinthefile,assumingthatyoudidn'texceed thesizeofthedevice.Remember,usingthismethod,thesizeisfixedatcreationtimeandcannotbe changed. 9.11.ChapterSummary •Partitionsarethelogicaldivisionofaphysicaldevice.Numerouspartitiontypesaresupported underLinux. •AfilesystemismountedonamountpointinLinux.Therootfilesystemismountedattheroot ofthefilesystemhierarchyandreferredtoas/. •Thepopularext2filesystemismatureandfast,andisoftenfoundonembeddedandotherLinux systemssuchasRedHatandtheFedoraCoreseries. •Theext3filesystemaddsjournalingontopoftheext2filesystem,forbetterdataintegrityand systemreliability. • ReiserFS is another popular and high-performance journaling file system found on many embeddedandotherLinuxsystems. • JFFS2 is a journaling file system optimized for use with Flash memory. It contains FlashfriendlyfeaturessuchaswearlevelingforlongerFlashmemorylifetime. • cramfs is a read-only file system perfect for small-system boot ROMs and other read-only programsanddata. •NFSisoneofthemostpowerfuldevelopmenttoolsfortheembeddeddeveloper.Itcanbring thepowerofaworkstationtoyourtargetdevice.LearnhowtouseNFSasyourembeddedtarget's rootfilesystem.Theconvenienceandtimesavingswillbeworththeeffort. • Many pseudo file systems are available on Linux. A few of the more important ones are presentedhere,includingtheprocfilesystemandsysfs. • The RAM-based tmpfs file system has many uses for embedded systems. Its most significant improvement over traditional ramdisks is the capability to resize itself dynamically to meet operationalrequirements. 9.11.1.SuggestionsforAdditionalReading "DesignandImplementationoftheSecondExtendedFilesystem" RémyCard,TheodoreTs'o,andStephenTweedie FirstpublishedintheProceedingsoftheFirstDutchInternationalSymposiumonLinux Availableonhttp://e2fsprogs.sourceforge.net/ext2intro.html "ANon-TechnicalLookInsidetheEXT2FileSystem" RandyAppleton www.linuxgazette.com/issue21/ext2.html Whitepaper:RedHat'sNewJournalingFileSystem:ext3 MichaelK.Johnson www.redhat.com/support/wpapers/redhat/ext3/ ReiserFSHomePage www.namesys.com/ "JFFS:TheJournalingFlashFileSystem" DavidWoodhouse http://sources.redhat.com/jffs2/jffs2.pdf READMEfilefromcramfsproject Unsigned(assumedtobetheprojectauthor) http://sourceforge.net/projects/cramfs/ NFShomepage http://nfs.sourceforge.net The/procfilesystemdocumentation www.tldp.org/LDP/lkmpg/2.6/html/c712.htm FileSystemPerformance:TheSolarisOS,UFS,Linuxext3,andReiserFS Atechnicalwhitepaper DominicKay www.sun.com/software/whitepapers/solaris10/fs_performance.pdf Chapter10.MTDSubsystem The Memory Technology Devices (MTD) subsystem grew out of the need to support a wide varietyofmemory-likedevicessuchasFlashmemorychips.ManydifferenttypesofFlashchipsare available,alongwithnumerousmethodstoprogramthem,partlybecauseofthemanyspecializedand high-performance modes that are supported. The MTD layer architecture enables the separation of the low-level device complexities from the higher-layer data organization and storage formats that usememorydevices. Inthischapter,weintroducetheMTDsubsystemandprovidesomesimpleexamplesofitsuse. FirstwelookatwhatisrequiredofthekerneltosupportMTDservices.Weintroducesomesimple operationsonadevelopmentworkstationwithMTDenabled,asameanstounderstandthebasicsof thissubsystem.Inthischapter,weintegrateMTDandtheJFFS2filesystem. We next introduce the concept of partitions as they relate to the MTD layer. We examine the detailsofbuildingpartitionsfromabootloaderandhowtheyaredetectedbytheLinuxkernel.The chaptercontinueswithabriefintroductiontotheMTDutilities.Weconcludebyputtingitalltogether andbootingatargetboardusinganin-FlashJFFS2filesystemimage. 10.1.EnablingMTDServices To use MTD services, your kernel must be configured with MTD enabled. Many configuration options exist for MTD, some of which can be confusing. The best way to understand the myriad choicesissimplytobeginworkingwiththem.ToillustratethemechanicsoftheMTDsubsystemand how it fits in with the system, we begin with some very simple examples that you can perform on your Linux development workstation. Figure 10-1 shows the kernel configuration (invoked per the usual make ARCH=<arch> gconfig) necessary to enable the bare-minimum MTD functionality. Listing10-1displaysthe.configfileentriesresultingfromtheselectionsshowninFigure10-1. Listing10-1.BasicMTDConfigurationfrom.config CONFIG_MTD=y CONFIG_MTD_CHAR=y CONFIG_MTD_BLOCK=y CONFIG_MTD_MTDRAM=m CONFIG_MTDRAM_TOTAL_SIZE=8192 CONFIG_MTDRAM_ERASE_SIZE=128 TheMTDsubsystemisenabledviathefirstconfigurationoption,whichisselectedviathefirst checkboxshowninFigure10-1,MemoryTechnologyDevice(MTD)Support.Thenexttwoentries fromtheconfigurationshowninFigure10-1enablespecialdevice-levelaccesstotheMTDdevices, such as Flash memory, from user space. The first one (CONFIG_MTD_CHAR) enables character devicemodeaccess,essentiallyasequentialaccesscharacterizedbybyte-at-a-timesequentialread andwriteaccess.Thesecond(CONFIG_MTD_BLOCK)enablesaccesstotheMTDdeviceinblock devicemode,theaccessmethodusedfordiskdrives,inwhichblocksofmultiplebytesofdataare readorwrittenatatime.TheseaccessmodesallowtheuseoffamiliarLinuxcommandstoreadand writedatatotheFlashmemory,asyoushallshortlysee. Figure10-1.MTDconfiguration TheCONFIG_MTD_MTDRAMelementenablesaspecialtestdriverthatenablesustoexamine the MTD subsystem even if we don't have any MTD devices (such as Flash memory) available. Coupled with this configuration selection are two parameters associated with the RAM-based test driver:thedevicesizeandtheerasesize.Forthisexample,wehavespecified8192KBtotalsizeand 128KB erase size. The objective of this test driver is to emulate a Flash device, primarily to facilitateMTDsubsystemtestinganddevelopment.BecauseFlashmemoryisarchitectedusingfixedsizeeraseblocks,thetestdriveralsocontainstheconceptoferaseblocks.Youwillseehowthese parametersareusedshortly. 10.1.1.BuildingMTD MTD is included in any recent snapshot of the Linux kernel. However, if you need to take advantageofMTDfeaturesthathavebeenaddedsinceyourkernelversionwasreleased,youmust downloadandbuildtheMTDdriversandutilities.BecausetheMTDpackagecontainsbothkernel components and user space programs, it is useful to keep the MTD package in a separate project directoryandconnectittoyourkernelsourcetree.ThesimplestwaytointegratetheMTDandyour kernelsourcetree(s)istousethescriptsprovidedbytheMTDpackage. Download the MTD package from the location given at the end of this chapter. Unpack the archive into a directory of your choice using the tar utility. Enter the directory and run the patchkernel.shscript.Thisscriptprovidesseveraloptions.Executethescriptwithnoparametersfor adetailedusage.Listing10-2showshowtoinstallthekernelcomponents. Listing10-2.PatchingYourKernelforMTD $./patchkernel.sh-2../sources/linux-2.6.10-mtd Patching../sources/linux-2.6.10-mtd/ IncludeJFFS2filesystem:jffs2 IncludeJFFS3filesystem(experimental):no Method:ln<<Willactuallycreatesymboliclinks Canwestartnow?[y/N]y $ Invoking the patchkernel.sh script with the -2 parameter indicates that we want support for the JFFS2filesystem.Weprovidethepathtothekernelsourcedirectoryas../sources/linux-2.6.10-mtd. Bydefault,patchkernel.shdoesnotcopyanyfilesintothekernelsourcedirectory.Instead,itcreates symboliclinksfromthekernelsourcetreepointingintotheMTDsubdirectoryitself.Inthisway,you canmaintainacommonsourcetreeforMTDforanynumberofkernelsthatyouhappentohaveon yourdevelopmentworkstation.ThisallowstheMTDkerneldriverstobebuiltwiththekernelbuild system,includinginformationaboutyourspecifickernelconfiguration. 10.2.MTDBasics NowthatwehaveenabledasimpleMTDconfigurationinourkernel,wecanexaminehowthis subsystem works on our Linux development workstation. Using the test RAM driver we just configured in the previous section, we can mount a JFFS2 image using an MTD device. Assuming thatyoucreatedaJFFS2imageasdetailedinChapter9,"FileSystems,"youmightwanttomountit andexamineit.TheLinuxkerneldoesnotsupportmountingaJFFS2filesystemimagedirectlyona loopback device, such as is possible with ext2 and other file system images. So we must use a differentmethod.ThiscanbeachievedusingtheMTDRAMtestdriveronourdevelopmentLinux workstationwithMTDenabled,asinFigure10-1.Listing10-3illustratesthesteps. Listing10-3.MountingJFFS2onanMTDRAMDevice #modprobejffs2 #modprobemtdblock #modprobemtdram #ddif=jffs2.binof=/dev/mtdblock0 4690+1recordsin 4690+1recordsout #mkdir/mnt/flash #mount-tjffs2/dev/mtdblock0/mnt/flash #ls-l/mnt/flash total0 drwxr-xr-x2rootroot0Sep1722:02bin drwxr-xr-x2rootroot0Sep1721:59dev drwxr-xr-x7rootroot0Sep1715:31etc drwxr-xr-x2rootroot0Sep1715:31home drwxr-xr-x2rootroot0Sep1722:02lib drwxr-xr-x2rootroot0Sep1715:31proc drws------2rootroot0Sep1715:31root drwxr-xr-x2rootroot0Sep1722:02sbin drwxrwxrwt2rootroot0Sep1715:31tmp drwxr-xr-x9rootroot0Sep1715:31usr drwxr-xr-x14rootroot0Sep1715:31var # FromListing10-3,firstweinstalltheloadablemodulesthattheLinuxkernelrequirestosupport JFFS2andtheMTDsubsystem.WeloadtheJFFS2modulefollowedbythemTDblockandmtdram modules.Afterthenecessarydevicedriversareloaded,weusetheLinuxddcommandtocopyour JFFS2filesystemimageintotheMTDRAMtestdriverusingthemTDblockdevice.Inessence,we areusingsystemRAMasabackingdevicetoemulateanMTDblockdevice. AfterwehavecopiedourJFFS2filesystemimageintotheMTDblockdevice,wecanmountit usingthemountcommand,inthemannershowninListing10-3.AftertheMTDpseudo-devicehas been mounted, we can work with the JFFS2 file system image in any way we choose. The only limitationusingthismethodisthatwecan'tenlargetheimage.Thesizeoftheimageislimitedbytwo factors.First,whenweconfiguredtheMTDRAMtestdevice,wegaveitamaximumsizeof8MB. [75]Second,whenwecreatedtheJFFS2image,wefixedthesizeoftheimageusingthemkfs.jffs2 utility.Theimagesizewasdeterminedbythecontentsofthedirectorywespecifiedwhenwecreated it.ReferbacktoListing9-9,inChapter9,torecallhowourjffs2.binimagewasbuilt. ItisimportanttorealizethelimitationsofusingthismethodtoexaminethecontentsofaJFFS2 file system. Consider what we did: We copied the contents of a file (the JFFS2 file system binary image) into a kernel block device (/dev/mtdblock0). Then we mounted the kernel block device (/dev/mtdblock)asaJFFS2filesystem.Afterwedidthis,wecoulduseallthetraditionalfilesystem utilitiestoexamineandevenmodifythefilesystem.Toolssuchasls,df,dh,mv,rm,andcpcanall be used to examine and modify the file system. However, unlike the loopback device, there is no connectionbetweenthefilewecopiedandthemountedJFFS2filesystemimage.Therefore,ifwe unmount the file system after making changes, the changes will be lost. If you want to save the changes,youmustcopythembackintoafile.Onesuchmethodisthefollowing: #ddif=/dev/mtdblock0of=./your-modified-fs-image.bin This command creates a file called your-modified-fs-image.bin that is the same size as the mtdblock0 device which was specified during configuration. In our example, it would be 8MB. Lacking suitable JFFS2 editing facilities, this is a perfectly valid way to examine and modify a JFFS2 file system. More important, it illustrates the basics of the MTD subsystem on our developmentsystemwithoutrealFlashmemory.Nowlet'slookatsomehardwarethatcontainsFlash physicaldevices. 10.2.1.ConfiguringMTD To use MTD with the Flash memory on your board, you must have MTD configured correctly. ThefollowinglistcontainstherequirementsthatmustbesatisfiedtoconfigureMTDforyourboard, Flash,andFlashlayout. •SpecifythepartitioningonyourFlashdevice •SpecifythetypeofFlashandlocation •ConfiguretheproperFlashdriverforyourchosenchip •Configurethekernelwiththeappropriatedriver(s) Eachofthesestepsisexploredinthefollowingsections. 10.3.MTDPartitions Most Flash devices on a given hardware platform are divided into several sections, called partitions, similar to the partitions found on a typical desktop workstation hard drive. The MTD subsystem provides support for such Flash partitions. The MTD subsystem must be configured for MTD partitioning support. Figure 10-2 illustrates the configuration options for MTD partitioning support. Figure10-2.KernelconfigurationforMTDpartitioningsupport Several methods exist for communicating the partition data to the Linux kernel. The following methodsarecurrentlysupported.YoucanseetheconfigurationoptionsforeachinFigure10-2under MTDPartitioningSupport. •Redbootpartitiontableparsing •Kernelcommand-linepartitiontabledefinition •Board-specificmappingdrivers MTDalsoallowsconfigurationswithoutpartitiondata.Inthiscase,MTDsimplytreatstheentire Flashmemoryasasingledevice. 10.3.1.RedbootPartitionTablePartitioning OneofthemorecommonmethodsofdefininganddetectingMTDpartitionsstemsfromoneofthe original implementations: Redboot partitions. Redboot is a bootloader found on many embedded boards,especiallyARMXScaleboardssuchastheADIEngineeringCoyoteReferencePlatform. TheMTDsubsystemdefinesamethodforstoringpartitioninformationontheFlashdeviceitself, similar in concept to a partition table on a hard disk. In the case of the Redboot partitions, the developerreservesandspecifiesaFlasheraseblockthatholdsthepartitiondefinitions.Amapping driverisselectedthatcallsthepartitionparsingfunctionsduringboottodetectthepartitionsonthe Flashdevice.Figure10-2showsthemappingdriverforourexampleboard;itisthefinalhighlighted entrydefiningCONFIG_MTD_IXP4xx. As usual, taking a detailed look at an example helps to illustrate these concepts. We start by lookingattheinformationprovidedbytheRedbootbootloaderfortheCoyoteplatform.Listing10-4 capturessomeoftheoutputoftheRedbootbootloaderuponpower-up. Listing10-4.RedbootMessagesonPower-Up Platform:ADICoyote(XScale) IDE/ParallelPortCPLDVersion:1.0 Copyright(C)2000,2001,2002,RedHat,Inc. RAM:0x00000000-0x04000000,0x0001f960-0x03fd1000available FLASH:0x50000000-0x51000000,128blocksof0x00020000byteseach. ... ThistellsusthatRAMonthisboardisphysicallymappedstartingataddress0x00000000and thatFlashismappedatphysicaladdress0x50000000through0x51000000.Wecanalsoseethatthe Flashhas128blocksof0x00020000(128KB)each. RedbootcontainsacommandtocreateanddisplaypartitioninformationontheFlash.Listing105 contains the output of the fis list command, part of the Flash Image System family of commands availableintheRedbootbootloader. Listing10-5.RedbootFlashPartitionList RedBoot>fislist NameFLASHaddrMemaddrLengthEntrypoint RedBoot0x500000000x500000000x000600000x00000000 RedBootconfig0x50FC00000x50FC00000x000010000x00000000 FISdirectory0x50FE00000x50FE00000x000200000x00000000 RedBoot> FromListing10-5,weseethattheCoyoteboardhasthreepartitionsdefinedontheFlash.The partition named RedBoot contains the executable Redboot bootloader image. The partition named RedBoot config contains the configuration parameters maintained by the bootloader. The final partitionnamedFISdirectoryholdsinformationaboutthepartitiontableitself. Whenproperlyconfigured,theLinuxkernelcandetectandparsethispartitiontableandcreate MTDpartitionsrepresentingthephysicalpartitionsonFlash.Listing10-6reproducesaportionofthe boot messages that are output from the aforementioned ADI Engineering Coyote board, booting a LinuxkernelconfiguredwithsupportfordetectingRedbootpartitions. Listing10-6.DetectingRedbootPartitionsonLinuxBoot ... IXP4XX-Flash0:Found1x16devicesat0x0in16-bitbank Intel/SharpExtendedQueryTableat0x0031 Usingbufferwritemethod cfi_cmdset_0001:Erasesuspendonwriteenabled SearchingforRedBootpartitiontableinIXP4XX-Flash0atoffset0xfe0000 3RedBootpartitionsfoundonMTDdeviceIXP4XX-Flash0 Creating3MTDpartitionson"IXP4XX-Flash0": 0x00000000-0x00060000:"RedBoot" 0x00fc0000-0x00fc1000:"RedBootconfig" 0x00fe0000-0x01000000:"FISdirectory" ... The first message in Listing 10-6 is printed when the Flash chip is detected, via the Common Flash Interface (CFI) driver, enabled via CONFIG_MTD_CFI. CFI is an industry-standard method for determining the Flash chip's characteristics, such as manufacturer, device type, total size, and eraseblocksize.SeeSection10.5.1,"SuggestionsforAdditionalReading,"attheendofthischapter forapointertotheCFIspecification. CFIisenabledviathekernel-configurationutilityundertheMemoryTechnologyDevices(MTD) top-level menu. Select Detect flash chips by Common Flash Interface (CFI) probe under RAM/ROM/Flashchipdrivers,asillustratedinFigure10-3. Figure10-3.KernelconfigurationforMTDCFIsupport As shown in Listing 10-6, the Flash chip is detected via the CFI interface. Because we also enabled CONFIG_MTD_REDBOOT_PARTS (see Figure 10-2), MTD scans for the Redboot partitiontableontheFlashchip.Noticealsothatthechiphasbeenenumeratedwiththedevicename IXP4XX-Flash0.YoucanseefromListing10-6thattheLinuxkernelhasdetectedthreepartitionson theFlashchip,asenumeratedpreviouslyusingthefislistcommandinRedboot. Whentheinfrastructureisinplaceasdescribedhere,theLinuxkernelautomaticallydetectsand createskerneldatastructuresrepresentingthethreeFlashpartitions.Evidenceofthesecanbefound inthe/procfilesystemwhenthekernelhascompletedinitialization,asshowninListing10-7. Listing10-7.KernelMTDFlashPartitions root@coyote:~#cat/proc/mtd dev:sizeerasesizename mtd0:0006000000020000"RedBoot" mtd1:0000100000020000"RedBootconfig" mtd2:0002000000020000"FISdirectory" # We can easily create a new Redboot partition. We use the Redboot FIS commands for this example,butwedonotdetailtheRedbootcommandsinthisbook.However,theinterestedreader canconsulttheRedbootuserdocumentationlistedinSection10.5.1attheendofthischapter.Listing 10-8showsthedetailsofcreatinganewRedbootpartition. Listing10-8.CreatingaNewRedbootPartition RedBoot>load-r-v-b0x01008000coyote-40-zImage Usingdefaultprotocol(TFTP) Rawfileloaded0x01008000-0x0114dccb,assumedentryat0x01008000 RedBoot>fiscreate-b0x01008000-l0x145cd0-f0x50100000MyKernel ...Erasefrom0x50100000-0x50260000:........... ...Programfrom0x01008000-0x0114dcd0at0x50100000:........... ...Unlockfrom0x50fe0000-0x51000000:. ...Erasefrom0x50fe0000-0x51000000:. ...Programfrom0x03fdf000-0x03fff000at0x50fe0000:. ...Lockfrom0x50fe0000-0x51000000:. First,weloadtheimagewewillusetocreatethenewpartition.Wewilluseourkernelimagefor theexample.Weloadittomemoryaddress0x01008000.Thenwecreatethenewpartitionusingthe Redbootfiscreatecommand.WehaveinstructedRedboottocreatethenewpartitioninanareaof Flashstartingat0x50100000.YoucanseetheactionasRedbootfirsterasesthisareaofFlashand thenprogramsthekernelimage.Inthefinalsequence,Redbootunlocksitsdirectoryareaandupdates theFISDirectorywiththenewpartitioninformation.Listing10-9showstheoutputoffislistwiththe newpartition.ComparethiswiththeoutputinListing10-5. Listing10-9.NewRedbootPartitionList RedBoot>fislist NameFLASHaddrMemaddrLengthEntrypoint RedBoot0x500000000x500000000x000600000x00000000 RedBootconfig0x50FC00000x50FC00000x000010000x00000000 FISdirectory0x50FE00000x50FE00000x000200000x00000000 MyKernel0x501000000x501000000x001600000x01008000 Ofcourse,whenweboottheLinuxkernel,itdiscoversthenewpartitionandwecanoperateonit asweseefit.Theastutereadermighthaverealizedtheotherbenefitofthisnewpartition:Wecan now boot the kernel from Flash instead of having to load it via tftp every time. The command is illustrated next. Simply pass the Redboot exec command the Flash starting address of the partition andthelengthoftheimagetotransferintoRAM. ...RedBoot>exec-b0x50100000-l0x145cd0 UncompressingLinux...........done,bootingthekernel. ... 10.3.2.KernelCommandLinePartitioning As detailed in Section 10.3, "MTD Partitions," the raw Flash partition information can be communicated to the kernel using other methods. Indeed, possibly the most straightforward, though perhapsnotthesimplestmethodistomanuallypassthepartitioninformationdirectlyonthekernel commandline.Ofcourse,aswehavealreadylearned,somebootloadersmakethateasy(forexample U-Boot),whereasothersdonothaveafacilitytopassakernelcommandlinetothekerneluponboot. Inthesecases,thekernelcommandlinemustbeconfiguredatcompiletimeand,therefore,ismore difficulttochange,requiringarecompileofthekernelitselfeachtimethepartitionsaremodified. Toenablecommand-linepartitioningintheMTDsubsystem,yourkernelmustbeconfiguredfor thissupport.YoucanseethisconfigurationoptioninFigure10-2underMTDpartitioningsupport. Select the option for command-line partition table parsing, which defines the CONFIG_MTD_CMDLINE_PARTSoption. Listing10-10showstheformatfordefiningapartitiononthekernelcommandline(takenfrom .../drivers/mtd/cmdlinepart.c). Listing10-10.KernelCommand-LineMTDPartitionFormat mtdparts=<mtddef>[;<mtddef] *<mtddef>:=<mtd-id>:<partdef>[,<partdef>] *<partdef>:=<size>[@offset][<name>][ro] *<mtd-id>:=uniquenameusedinmappingdriver/device(mtd->name) *<size>:=stdlinuxmemsizeOR"-"todenoteallremainingspace *<name>:='('NAME')' Eachmtddefparameterpassedonthekernelcommandlinedefinesaseparatepartition.Asshown is Listing 10-10, each mtddef definition contains multiple parts. You can specify a unique ID, partition size, and offset from the start of the Flash. You can also pass the partition a name and, optionally,theread-onlyattribute.ReferringbacktoourRedbootpartitiondefinitionsinListing10-5, wecouldstaticallydefinetheseonthekernelcommandlineasfollows: mtdparts=MainFlash:384K(Redboot),4K(config),128K(FIS),-(unused) With this definition, the kernel would instantiate four MTD partitions, with an MTD ID of MainFlash,containingthesizesandlayoutmatchingthatfoundinListing10-5. 10.3.3.MappingDriver The final method for defining your board-specific Flash layout is to use a dedicated boardspecificmappingdriver.TheLinuxkernelsourcetreecontainsmanyexamplesofmappingdrivers, locatedin.../drivers/mtd/maps.Anyoneofthesewillprovidegoodexamplesforhowtocreateyour own.Theimplementationdetailsvarybyarchitecture. Themappingdriverisaproperkernelmodule,completewithmodule_init()andmodule_exit() calls,asdescribedinChapter8,"DeviceDriverBasics."Atypicalmappingdriverissmallandeasy tonavigate,oftencontainingfewerthanacoupledozenlinesofC. Listing10-11reproducesasectionof.../drivers/mtd/maps/pq2fads.Thismappingdriverdefines theFlashdeviceonaFreescalePQ2FADSevaluationboardthatsupportstheMPC8272andother processors. Listing10-11.PQ2FADsFlashMappingDriver ... staticstructmtd_partitionpq2fads_partitions[]={ { #ifdefCONFIG_ADS8272 .name="HRCW", .size=0x40000, .offset=0, .mask_flags=MTD_WRITEABLE,/*forceread-only*/ },{ .name="UserFS", .size=0x5c0000, .offset=0x40000, #else .name="UserFS", .size=0x600000, .offset=0, #endif },{ .name="uImage", .size=0x100000, .offset=0x600000, .mask_flags=MTD_WRITEABLE,/*forceread-only*/ },{ .name="bootloader", .size=0x40000, .offset=0x700000, .mask_flags=MTD_WRITEABLE,/*forceread-only*/ },{ .name="bootloaderenv", .size=0x40000, .offset=0x740000, .mask_flags=MTD_WRITEABLE,/*forceread-only*/ } }; /*pointertoMPC885ADSboardinfodata*/ externunsignedchar__res[]; staticint__initinit_pq2fads_mtd(void){ bd_t*bd=(bd_t*)__res; physmap_configure(bd->bi_flashstart,bd->bi_flashsize,PQ2FADS_BANK_WIDTH,NULL); physmap_set_partitions(pq2fads_partitions, sizeof (pq2fads_partitions) / sizeof (pq2fads_partitions[0])); return0; } staticvoid__exitcleanup_pq2fads_mtd(void){} module_init(init_pq2fads_mtd); module_exit(cleanup_pq2fads_mtd); ... ThissimplebutcompleteLinuxdevicedrivercommunicatesthePQ2FADSFlashmappingtothe MTDsubsystem.RecallfromChapter8thatwhenafunctioninadevicedriverisdeclaredwiththe module_init()macro,itisautomaticallyinvokedduringLinuxkernelbootattheappropriatetime.In this PQ2FADS mapping driver, the module initialization function init_pq2fads_mtd() performs just twosimplecalls: •physmap_configure()passestotheMTDsubsystemtheFlashchip'sphysicaladdress,size,and bankwidth,alongwithanyspecialsetupfunctionrequiredtoaccesstheFlash. •physmap_set_partitions()passestheboard'suniquepartitioninformationtotheMTDsubsystem from the partition table defined in the pq2fads_partitions[] array found at the start of this mapping driver. Followingthissimpleexample,youcanderiveamappingdriverforyourownboard. 10.3.4.FlashChipDrivers MTDhassupportforawidevarietyofFlashchipsanddevices.Chancesareverygoodthatyour chosen chip has also been supported. The most common Flash chips support the Common Flash Interface(CFI)mentionedearlier.OlderFlashchipsmighthaveJEDECsupport,whichisanolder Flashcompatibilitystandard.Figure10-4showsthekernelconfigurationfromarecentLinuxkernel snapshot.ThisversionsupportsmanyFlashtypes. Figure10-4.Flashdevicesupport If your Flash chip is not supported, you must provide a device file yourself. Using one of the manyexamplesin.../drivers/mtd/chipsasastartingpoint,customizeorcreateyourownFlashdevice driver.Betteryet,unlessthechipwasjustintroducedwithsomenewfangledinterface,chancesare goodthatsomeonehasalreadyproducedadriver. 10.3.5.Board-SpecificInitialization Alongwithamappingdriver,yourboard-specific(platform)setupmustprovidetheunderlying definitionsforproperMTDFlashsystemoperation.Listing10-12reproducestherelevantportions of.../arch/arm/mach-ixp4xx/coyote-setup.c. Listing10-12.Coyote-SpecificBoardSetup staticstructflash_platform_datacoyote_flash_data={ .map_name="cfi_probe", .width=2, }; staticstructresourcecoyote_flash_resource={ .start=COYOTE_FLASH_BASE, .end=COYOTE_FLASH_BASE+COYOTE_FLASH_SIZE-1, .flags=IORESOURCE_MEM, }; staticstructplatform_devicecoyote_flash={ .name="IXP4XX-Flash", .id=0, .dev={ .platform_data=&coyote_flash_data, }, .num_resources=1, .resource=&coyote_flash_resource, }; ... staticstructplatform_device*coyote_devices[]__initdata={ &coyote_flash, &coyote_uart }; staticvoid__initcoyote_init(void){ ... platform_add_devices(coyote_devices,ARRAY_SIZE(coyote_devices)); } ... InListing 10-12, onlytherelevantportionsofthecoyote-setup.cplatforminitialization file are reproduced. Starting from the bottom, the coyote_init() function calls platform_add_devices(), specifyingtheCoyote-specificdevicesdefinedearlierinthisfile.You'llnoticethattwodevicesare defined just above the coyote_init() routine. The one we're interested in for this discussion is coyote_flash.Thisstructureoftypestructplatform_devicecontainsalltheimportantdetailsneeded bytheLinuxkernelandMTDsubsystem. The.namememberofthecoyote_flashstructurebindsourplatform-specificFlashresourcetoa mapping driver with the same name. You can see this in the mapping driver file .../drivers/mtd/maps/ixp4xx.c.The.resourcemembercommunicatesthebaseaddressoftheFlashon theboard.The.devmember,whichcontainsa.platform_datamember,tiesourFlashsetuptoachip driver.Inthiscase,wehavespecifiedthatourboardwillusetheCFIprobemethod,specifiedinthe kernelconfigurationasCONFIG_MTD_CFI.YoucanseethisconfigurationselectioninFigure10-4. Dependingonyourownarchitectureandboard,youcanuseamethodsimilartothistodefinethe Flashsupportforyourownboard. 10.4.MTDUtilities TheMTDpackagecontainsanumberofsystemutilitiesusefulforsettingupandmanagingyour MTDsubsystem.TheutilitiesarebuiltseparatelyfromtheprimaryMTDsubsystem,whichshouldbe builtfromwithinyourLinuxkernelsourcetree.Theutilitiescanbebuiltinasimilarmannertoany othercross-compileduserspacecode. Youmustusecautionwhenusingtheseutilitiesbecausethereisnoprotectionfrommistakes.A single-digittypocanwipeoutthebootloaderonyourhardwareplatform,whichcandefinitelyruin yourdayunlessyou'vebackeditupandknowhowtoreprogramitusingaJTAGFlashprogrammer. In keeping with a common practice throughout this book, we cannot devote sufficient space to covereveryMTDutility.Wehighlightthemostcommonandusefulones,andleaveitasanexercise forthereadertoexploretherest.ArecentMTDsnapshotcontainedmorethan20binaryutilities. The flash_* family of utilities is useful for raw device operations on an MTD partition. These includeflashcp,flash_erase,flash_info,flash_lock,flash_unlock,andothers.Hopefullytheirnames are descriptive enough to give some idea of their function. After partitions are defined and enumeratedaskerneldevices,anyoftheseuserspaceutilitiescanberunonapartition.Werepeat thewarningweissuedearlier:Ifyouexecuteflash_eraseonthepartitioncontainingyourbootloader, you'llbetheproudownerofasiliconpaperweight.Ifyouintendtoexperimentlikethis,it'sagood ideatohaveabackupofyourbootloaderimageandknowhowtore-FlashitusingahardwareJTAG emulatororotherFlashprogrammingtool. Our new partition created in Listing 10-8 (MyKernel) shows up in the kernel running on the Coyote board, as detailed in Listing 10-13. Here you can see the new partition we created instantiatedasthekerneldevicemTD1. Listing10-13.KernelMTDPartitionList root@coyote:~#cat/proc/mtd dev:sizeerasesizename mtd0:0006000000020000"RedBoot" mtd1:0016000000020000"MyKernel" mtd2:0000100000020000"RedBootconfig"x mtd3:0002000000020000"FISdirectory" UsingtheMTDutilities,wecanperformanumberofoperationsonthenewlycreatedpartition. Thefollowingshowstheresultsofaflash_erasecommandonthepartition: #flash_erase/dev/mtd1 EraseTotal1Units PerformingFlashEraseoflength131072atoffset0x0done Tocopyanewkernelimagetothispartition,usetheflashcpcommand: root@coyote:~#flashcp/workspace/coyote-40-zImage/dev/mtd1 Itgetsabitmoreinterestingworkingwitharootfilesystempartition.Wehavetheoptionofusing thebootloaderortheLinuxkerneltoplacetheinitialimageontheRedbootflashpartition.First,we useRedboottocreatethenewpartitionthatwillholdourrootfilesystem.Thefollowingcommand createsanewpartitionontheFlashcalledRootFSstartingatphysicalmemory0x50300000,witha length of 30 blocks. Remember, a block, generically called an erase unit, is 128KB on this Flash chip. RedBoot>fiscreate-f0x50300000-l0x600000-nRootFS Next, we boot the kernel and copy the root file system image into the new partition we have namedRootFS.ThisisaccomplishedwiththefollowingcommandfromaLinuxcommandprompton your target board. Note that this assumes you have already placed your file system image in a directoryaccessibletoyourboard.Asmentionedmanytimesthroughoutthisbook,NFSisyourbest choicefordevelopment. root@coyote:~#flashcp/rootfs.ext2/dev/mtd2 Thefilesystemcanbeanywherefromacouplemegabytesuptothelargestsizewehaveallowed on this partition, so this can take some time. Remember, this operation involves programming (sometimes called flashing) the image into the Flash memory. After copying, we can mount the partitionasafilesystem.Listing10-14displaysthesequence. Listing10-14.MountingMTDFlashPartitionasext2FileSystem root@coyote:~#mount-text2/dev/mtdblock2/mnt/remotero root@coyote:~#ls-l/mnt/remote/ total16 drwxr-xr-x2rootroot1024Nov192005bin drwxr-xr-x2rootroot1024Oct262005boot drwxr-xr-x2rootroot1024Nov192005dev drwxr-xr-x5rootroot1024Nov192005etc drwxr-xr-x2rootroot1024Oct262005home drwxr-xr-x3rootroot1024Nov192005lib drwxr-xr-x3rootroot1024Nov192005mnt drwxr-xr-x2rootroot1024Oct262005opt drwxr-xr-x2rootroot1024Oct262005proc drwxr-xr-x2rootroot1024Oct262005root drwxr-xr-x2rootroot1024Nov192005sbin drwxr-xr-x2rootroot1024Oct262005srv drwxr-xr-x2rootroot1024Oct262005sys drwxr-xr-x2rootroot1024Oct262005tmp drwxr-xr-x6rootroot1024Oct262005usr drwxr-xr-x2rootroot1024Nov192005var root@coyote:~# Listing10-14hastwoimportantsubtleties.Noticethatwehavespecified/dev/mtdblock2onthe mountcommandline.ThisistheMTDblockdriverthatenablesustoaccesstheMTDpartitionasa block device. Using /dev/mtd2 as a specifier instructs the kernel to use the MTD character driver. Boththemtdcharandmtdblockarepseudodriversusedtoprovideeithercharacter-basedorblockoriented access to the underlying Flash partition. Because mount expects a block device, you must usetheblock-devicespecifier.Figure10-1showsthekernelconfigurationthatenablestheseaccess methods. The respective kernel configuration macros are CONFIG_MTD_CHAR and CONFIG_MTD_BLOCK. Thesecondsubtletyistheuseoftheread-only(ro)command-lineswitchonthemountcommand. Itisperfectlyacceptabletomountanext2imagefromFlashusingtheMTDblockemulationdriver forread-onlypurposes.However,thereisnosupportforwritingtoanext2deviceusingthemtdblock driver.Thisisbecauseext2hasnoknowledgeofFlasheraseblocks.ForwriteaccesstoaFlashbasedfilesystem,weneedtouseafilesystemwithFlashknowledge,suchasJFFS2. 10.4.1.JFFS2RootFileSystem Creating a JFFS2 root file system is a straightforward process. In addition to compression, JFFS2supportswearleveling,afeaturedesignedtoincreaseFlashlifetimebyfairlydistributingthe writecyclesacrosstheblocksofthedevice.AspointedoutinChapter9,Flashmemoryissubjectto a limited number of write cycles. Wear leveling should be considered a mandatory feature in any Flash-basedfilesystemyouemploy.Asmentionedelsewhereinthisbook,youshouldconsiderFlash memoryasawrite-occasionalmedium.Specifically,youshouldavoidallowinganyprocessesthat requirefrequentwritestotargettheFlashfilesystem.Beespeciallyawareofanyloggingprograms, suchassyslogd. WecanbuildaJFFS2imageonourdevelopmentworkstationusingtheext2imageweusedon ourRedbootRootFSpartition.Thecompressionbenefitswillbeimmediatelyobvious.Theimagewe usedinthepreviousRootFSexamplewasanext2filesystemimage.Hereisthelistinginlong(-l) format: #ls-lrootfs.ext2 -rw-r--r--1rootroot6291456Nov1916:21rootfs.ext2 Nowlet'sconvertthisfilesystemimagetoJFFS2usingthemkfs.jffs2utilityfoundintheMTD package.Listing10-15showsthecommandandresults. Listing10-15.ConvertingRootFStoJFFS2 #mount-olooprootfs.ext2/mnt/flash/ #mkfs.jffs2-r/mnt/flash-e128-b-orootfs.jffs2 #ls-lrootfs.jffs2 -rw-r--r--1rootroot2401512Nov2010:08rootfs.jffs2 # Firstwemounttheext2filesystemimageonaloopbackdeviceonanarbitrarymountpointon our development workstation. Next we invoke the MTD utility mkfs.jffs2 to create the JFFS2 file systemimage.The-rflagtellsmkfs.jffs2wheretherootfilesystemimageislocated.The-einstructs mkfs.jffs2tobuildtheimagewhileassuminga128KBblocksize.Thedefaultis64KB.JFFS2does notexhibititsmostefficientbehavioriftheFlashdevicecontainsadifferentblocksizethantheblock sizeoftheimage.Finally,wedisplayalonglistinganddiscoverthattheresultingJFFS2rootfile systemimagehasbeenreducedinsizebymorethan60percent.Whenyouareworkingwithlimited Flashmemory,thisisasubstantialreductioninpreciousFlashresourceusage. Takenoteofanimportantcommand-lineflagpassedtomkfs.jffs2inListing10-15.The-bflagis the-big-endian flag. Thisinstructsthe mkfs.jffs2utilitytocreateaJFFS2Flashimagesuitablefor use on a big-endian target. Because we are targeting the ADI Engineering Coyote board, which contains an Intel IXP425 processor running in big-endian mode, this step is crucial for proper operation.Ifyoufailtospecifybigendian,youwillgetseveralscreensfullofcomplaintsfromthe kernelasittriestonegotiatethesuperblockofaJFFS2filesystemthatisessentiallygibberish.[76] AnyonecaretoguesshowIrememberedthisimportantdetail? In a similar manner to the previous example, we can copy this image to our Redboot RootFS Flashpartitionusingtheflashcputility.ThenwecanboottheLinuxkernelusingaJFFS2rootfile system.Listing10-16providesthedetails,runningtheMTDutilitiesonourtargethardware. Listing10-16.CopyingJFFS2toRootFSPartition root@coyote:~#cat/proc/mtd dev:sizeerasesizename mtd0:0006000000020000"RedBoot" mtd1:0016000000020000"MyKernel" mtd2:0060000000020000"RootFS" mtd3:0000100000020000"RedBootconfig" mtd4:0002000000020000"FISdirectory" root@coyote:~#flash_erase/dev/mtd2 EraseTotal1Units PerformingFlashEraseoflength131072atoffset0x0done root@coyote:~#flashcp/rootfs.jffs2/dev/mtd2 root@coyote:~# It is important to note that you must have the JFFS2 file system enabled in your kernel configuration. Execute make ARCH=<arch> gconfig and select JFFS2 under File Systems, MiscellaneousFileSystems.Anotherusefulhintistousethe-v(verbose)flagontheMTDutilities. ThisprovidesprogressupdatesandotherusefulinformationduringtheFlashoperations. We have already seen how to boot a kernel with the Redboot exec command. Listing 10-17 detailsthesequenceofcommandstoloadandboottheLinuxkernelwithournewJFFS2filesystem asroot. Listing10-17.BootingwithJFFS2asRootFileSystem RedBoot>load-r-v-b0x01008000coyote-zImage Usingdefaultprotocol(TFTP) Rawfileloaded0x01008000-0x0114decb,assumedentryat0x01008000 RedBoot>exec-c"console=ttyS0,115200rootfstype=jffs2root=/dev/mtdblock2" Usingbaseaddress0x01008000andlength0x00145ecc UncompressingLinux......done,bootingthekernel. ... 10.5.ChapterSummary •TheMemoryTechnologyDevices(MTD)subsystemprovidessupportformemorydevicessuch asFlashmemoryintheLinuxkernel. • MTD must be enabled in your Linux kernel configuration. Several figures in this chapter detailedtheconfigurationoptions. •AspartoftheMTDkernelconfiguration,theproperFlashdriver(s)foryourFlashchipsmust beselected.Figure10-4presentedthecollectionofchipdriverssupportedinarecentLinuxkernel snapshot. • Your Flash memory device can be managed as a single large device or can be divided into multiplepartitions. •SeveralmethodsareavailableforcommunicatingthepartitioninformationtotheLinuxkernel. TheseincludeRedbootpartitioninformation,kernelcommand-lineparameters,andmappingdrivers. • A mapping driver, together with definitions supplied by your architecture-specific board support,definesyourFlashconfigurationtothekernel. •MTDcomeswithanumberofuserspaceutilitiestomanagetheimagesonyourFlashdevices. • The Journaling Flash File System 2 (JFFS2) is a good companion to the MTD subsystem for small,efficientFlash-basedfilesystems.Inthischapter,webuiltaJFFS2imageandmounteditas rootonourtargetdevice. 10.5.1.SuggestionsforAdditionalReading MTDLinuxhomepage www.linux-mtd.infradead.org/ Redbootuserdocumentation http://ecos.sourceware.org/ecos/docs-latest/redboot/redboot-guide.html CommonFlashMemoryInterfaceSpecification AMDCorporation www.amd.com/us-en/assets/content_type/DownloadableAssets/cfi_r20.pdf Chapter11.BusyBox The man page for BusyBox declares that BusyBox is "The Swiss Army Knife of Embedded Linux." This is a fitting description, for BusyBox is a small and efficient replacement for a large collectionofstandardLinuxcommandlineutilities.Itoftenservesasthefoundationforaresourcelimitedembeddedplatform.ThischapterintroducesBusyBoxandprovidesagoodstartingpointfor customizingyourownBusyBoxinstallation. WepreviouslyalludedtoBusyBoxinmultiplelocations.Inthischapter,wepresentthedetailsof this useful package. After a brief introduction to BusyBox, we explore the BusyBox configuration utility. This is used to tailor BusyBox to your particular requirements. We then discuss the requirementsforcross-compilingtheBusyBoxpackage. BusyBoxoperationalissuesareconsidered,includinghowitisusedinanembeddedsystem.We examinetheBusyBoxinitializationsequenceandexplainhowthisdepartsfromthestandardSystem Vinitialization.Inthissection,wealsopresentanexampleinitializationscript.Afterseeingthesteps forinstallingBusyBoxonatargetsystem,youwilllearnaboutsomeoftheBusyBoxcommandsand theirlimitations. 11.1.IntroductiontoBusyBox BusyBoxhasgained tremendous popularityintheembeddedLinuxcommunity.It isremarkably easytoconfigure,compile,anduse,andithasthepotentialtosignificantlyreducetheoverallsystem resources required to support a wide collection of common Linux utilities. BusyBox provides compactreplacementsformanytraditionalfull-blownutilitiesfoundonmostdesktopandembedded Linuxdistributions.Examplesincludethefileutilitiessuchasls,cat,cp,dir,head,andtail;general utilities such as dmesg, kill, halt, fdisk, mount, umount; and many more. BusyBox also provides supportformorecomplexoperations,suchasifconfig,netstat,route,andothernetworkutilities. BusyBox is modular and highly configurable, and can be tailored to suit your particular requirements.ThepackageincludesaconfigurationutilitysimilartothatusedtoconfiguretheLinux kernelandwill,therefore,seemquitefamiliar. The commands in BusyBox are generally simpler implementations than their full-blown counterparts. In some cases, only a subset of the usual command line options is supported. In practice, however, you will find that the BusyBox subset of command functionality is more than sufficientformostgeneralembeddedrequirements. 11.1.1.BusyBoxisEasy If you are able to configure and build the Linux kernel, you will find BusyBox very straightforwardtoconfigure,build,andinstall.Thestepsaresimilar: 1.Executeaconfigurationutilityandenableyourchoiceoffeatures 2.Runmakedeptobuildadependencytree 3.Runmaketobuildthepackage 4.Installthebinaryandaseriesofsymboliclinks[77]onyourtargetsystem You can build and install BusyBox on your development workstation or your target embedded system. BusyBox works equally well in both environments. However, you must take care when installingonyourdevelopmentworkstationthatyoukeepitisolatedinaworkingdirectory,toavoid overwritingyoursystem'sstartupfilesorprimaryutilities. 11.2.BusyBoxConfiguration ToinitiatetheBusyBoxconfiguration,thecommandisthesameasthatusedwiththeLinuxkernel forthencurseslibrary-basedconfigurationutility: $makemenuconfig Figure11-1showsthetop-levelBusyBoxconfiguration. Figure11-1.Top-LevelBusyBoxConfigurationmenu Space does not permit coverage of each configuration option. However, some of the options deservemention.SomeofthemoreimportantBusyBoxconfigurationoptionsarefoundunderBuild Options. Here you will find configuration options necessary to cross-compile the BusyBox application.Listing11-1detailstheoptionsfoundunderBuildOptionsinarecentBusyBoxsnapshot. SelectBuildOptionsfromthetop-levelBusyBoxconfigurationutilitytonavigatetothisscreen. Listing11-1.BusyBoxBuildOptions []BuildBusyBoxasastaticbinary(nosharedlibs) []BuildwithLargeFileSupport(foraccessingfiles>2GB) []DoyouwanttobuildBusyBoxwithaCrossCompiler? ()AnyextraCFLAGSoptionsforthecompiler? Thefirstoptionisusefulforbuildingveryminimalembeddedsystems.ItallowsBusyBoxtobe compiled and linked statically so that no dynamically loaded libraries (libc-2.3.3.so, for example) arerequiredatruntimeonthetargetsystem.Withoutthisoption,BusyBoxrequiressomelibrariesso it can run. We can easily determine what libraries BusyBox (or any other binary) requires on our targetsystembyusingthelddcommand.Listing11-2containstheoutputasdisplayedonmydesktop Linuxworkstation. Listing11-2.BusyBoxLibraryDependencies $lddbusybox linux-gate.so.1=>(0xffffe000) libc.so.6=>/lib/tls/libc.so.6(0x42c70000) /lib/ld-linux.so.2=>/lib/ld-linux.so.2(0x42c57000) Notice that the BusyBox utility, as compiled using the default configuration, requires the three shared libraries in Listing 11-2. Had we elected to build BusyBox as a static binary, ldd would simply issue a message telling us that the BusyBox binary is not a dynamic executable. In other words, it requires no shared libraries to resolve any unresolved dependencies in the executable. Static linking yields a smaller footprint on a root file system because no shared libraries are required.However,buildinganembeddedapplicationwithoutsharedlibrariesmeansthatyouhave noneofthefamiliarClibraryfunctionsavailabletoyourapplications. WecovertheotheroptionsfromListing11-1inthenextsection. 11.2.1.Cross-CompilingBusyBox Asmentionedatthebeginningofthechapter,theauthorsofBusyBoxintendedthepackagetobe usedinacross-developmentenvironment,sobuildingBusyBoxinsuchanenvironmentisquiteeasy. Inmostcases,theonlyrequirementistospecifytheprefixtothecross-compileronyourdevelopment workstation.ThisisspecifiedinBuildOptionsintheBusyBoxconfigurationutilitybyselectingthe optiontobuildBusyBoxwithacross-compiler.Youthenarepresentedwithanoptiontoenterthe cross-compilerprefix.Theprefixyouenterdependsonyourcross-developmentenvironment.Some examplesincludexscale_be-orppc-linux-.Wecoverthisinmoredetailinthenextchapterwhenwe examinetheembeddeddevelopmentenvironment. ThefinaloptioninListing11-1isforanyextraflagsyoumightwanttoincludeonthecompiler commandline.Thesemightincludeoptionsforgeneratingdebuginformation(-g),optionsforsetting the optimization level (-O2, for example), and other compiler options that might be unique to your particularinstallationandtargetsystem. 11.3.BusyBoxOperation WhenyoubuildBusyBox,youendupwithabinarycalled,youguessedit,busybox.BusyBoxcan beinvokedfromthebinarynameitself,butitismoreusuallylaunchedviaasymlink.WhenBusyBox isinvokedwithoutcommandlineparameters,itproducesalistofthefunctionsthatwereenabledvia the configuration. Listing 11-3 shows such an output (it has been formatted slightly to fit the page width). Listing11-3.BusyBoxUsage root@coyote#./busybox BusyBoxv1.01(2005.12.03-18:00+0000)multi-callbinary Usage:busybox[function][arguments]... or:[function][arguments]... BusyBoxisamulti-callbinarythatcombinesmanycommonUnix utilitiesintoasingleexecutable.Mostpeoplewillcreatea linktobusyboxforeachfunctiontheywishtouseandBusyBox willactlikewhateveritwasinvokedas! Currentlydefinedfunctions: [,ash,basename,bunzip2,busybox,bzcat,cat,chgrp,chmod, chown,chroot,chvt,clear,cmp,cp,cut,date,dd,deallocvt, df,dirname,dmesg,du,echo,egrep,env,expr,false,fgrep, find,free,grep,gunzip,gzip,halt,head,hexdump,hostname, id,ifconfig,init,install,kill,killall,klogd,linuxrc,ln, logger,ls,mkdir,mknod,mktemp,more,mount,mv,openvt,pidof, ping,pivot_root,poweroff,ps,pwd,readlink,reboot,reset, rm,rmdir,route,sed,sh,sleep,sort,strings,swapoff,swapon, sync,syslogd,tail,tar,tee,test,time,touch,tr,true,tty, umount,uname,uniq,unzip,uptime,usleep,vi,wc,wget,which, whoami,xargs,yes,zcat FromListing11-3,youcanseethelistoffunctionsthatareenabledinthisBusyBoxbuild.They are listed in alphabetical order from ash (a shell optimized for small memory footprint) to zcat, a utilityusedtodecompressthecontentsofacompressedfile.Thisisthedefaultsetofutilitiesenabled inthisparticularBusyBoxsnapshot. Toinvokeaparticularfunction,executebusyboxwithoneofthedefinedfunctionspassedonthe commandline.Thus,todisplayalistingoffilesinthecurrentdirectory,executethiscommand: [root@coyote]#./busyboxls Another important message from the BusyBox usage message in Listing 11-3 is the short description of the program. It describes BusyBox as a multicall binary, combining many common utilitiesintoasingleexecutable.Thisisthepurposeofthesymlinksmentionedearlier.BusyBoxwas intendedtobeinvokedbyasymlinknamedforthefunctionitwillperform.Thisremovestheburden ofhavingtotypeatwo-wordcommandtoinvokeagivenfunction,anditpresentstheuserwithaset offamiliarcommandsforthesimilarlynamedutilities.Listings11-4and11-5shouldmakethisclear. Listing11-4.BusyBoxSymlinkStructureTopLevel [root@coyote]$ls-l/ total12 drwxrwxr-x2rootroot4096Dec313:38bin lrwxrwxrwx1rootroot11Dec313:38linuxrc->bin/busybox drwxrwxr-x2rootroot4096Dec313:38sbin drwxrwxr-x4rootroot4096Dec313:38usr Listing11-4showsthetargetdirectorystructureasbuiltbytheBusyBoxpackageviathemake installcommand.Theexecutablebusyboxfileisfoundinthe/bindirectory,andsymlinkshavebeen populatedthroughouttherestofthestructurepointingbackto/bin/busybox.Listing11-5expandson thedirectorystructureofListing11-4. Listing11-5.BusyBoxSymlinkStructureTreeDetail [root@coyote]$tree . |--bin ||--ash->busybox ||--busybox ||--cat->busybox ||--cp->busybox ||--... |'--zcat->busybox |--linuxrc->bin/busybox |--sbin ||--halt->../bin/busybox ||--ifconfig->../bin/busybox ||--init->../bin/busybox ||--klogd->../bin/busybox ||--... |'--syslogd->../bin/busybox '--usr |--bin ||--[->../../bin/busybox ||--basename->../../bin/busybox |--... ||--xargs->../../bin/busybox |'--yes->../../bin/busybox '--sbin '--chroot->../../bin/busybox TheoutputofListing11-5hasbeensignificantlytruncatedforreadabilityandtoavoidathreepagelisting.Eachlinecontaininganellipsis(...)indicatesthatthislistinghasbeenprunedtoshow onlythefirstfewandlastfewentriesofthatgivendirectory.Inactuality,morethan100symlinkscan be populated in these directories, depending on what functionality you have enabled using the BusyBoxconfigurationutility. Notice the busybox executable itself, the second entry from the /bin directory. Also in the /bin directory are symlinks pointing back to busybox for ash, cat, cp ... all the way to zcat. Again, the entries between cp and zcat have been omitted from this listing for readability. With this symlink structure,theusersimplyenterstheactualnameoftheutilitytoinvokeitsfunctionality.Forexample, to configure a network interface using the busybox ifconfig utility, the user might enter a command similartothis: $ifconfigeth1192.168.1.14 Thiswouldinvokethebusyboxexecutablethroughtheifconfigsymlink.BusyBoxexamineshow itwascalledthatis,itreadsargv[0]todeterminewhatfunctionalityisexecuted. 11.3.1.BusyBoxInit NoticethesymlinkinListing11-5calledinit.InChapter6"SystemInitialization,"youlearned abouttheinitprogramanditsroleinsysteminitialization.Recallthatthekernelattemptstoexecutea program called /sbin/init as the last step in kernel initialization. There is no reason why BusyBox can't emulate the init functionality, and that's exactly how the system illustrated by Listing 11-5 is configured.BusyBoxhandlestheinitfunctionality. BusyBox handles system initialization differently from standard System V init. A Linux system usingtheSystemV(SysV)initializationasdescribedinChapter6requiresaninittabfileaccessible inthe/etcdirectory.BusyBoxalsoreadsaninittabfile,butthesyntaxoftheinittabfileisdifferent.In general,youshouldnotneedtouseaninittabifyouareusingBusyBox.IagreewiththeBusyBoxman page:Ifyouneedrunlevels,useSystemVinitialization.[78] Let'sseewhatthislookslikeonanembeddedsystem.Wehavecreatedasmallrootfilesystem based on BusyBox. We configured BusyBox for static linking, eliminating the need for any shared libraries.Listing11-6containsatreelistingofthisrootfilesystem.Webuiltthissmallfilesystem usingthestepsoutlinedinChapter9,"FileSystems,"Section9.10,"BuildingaSimpleFileSystem." We do not detail the procedure again here. The files in our simple file system are those shown in Listing11-6. Listing11-6.MinimalBusyBoxRootFileSystem $tree . |--bin ||--busybox ||--cat->busybox ||--dmesg->busybox ||--echo->busybox ||--hostname->busybox ||--ls->busybox ||--ps->busybox ||--pwd->busybox |'--sh->busybox |--dev |'--console |--etc '--proc 4directories,10files ThisBusyBox-basedrootfilesystemoccupieslittlemorethanthesizeneededforbusyboxitself. Inthisconfiguration,usingstaticlinkingandsupportingnearly100utilities,theBusyBoxexecutable cameinatlessthan1MB: #ls-l/bin/busybox -rwxr-xr-x1rootroot824724Dec32005/bin/busybox Nowlet'sseehowthissystembehaves.Listing11-7capturestheconsoleoutputonpower-upon thisBusyBox-basedembeddedsystem. Listing11-7.BusyBoxDefaultStartup ... LookingupportofRPC100003/2on192.168.1.9 LookingupportofRPC100005/1on192.168.1.9 VFS:Mountedroot(nfsfilesystem). Freeinginitmemory:96K Bummer,couldnotrun'/etc/init.d/rcS':Nosuchfileordirectory PleasepressEntertoactivatethisconsole. BusyBoxv1.01(2005.12.03-19:09+0000)Built-inshell(ash) Enter'help'foralistofbuilt-incommands. -sh:can'taccesstty;jobcontrolturnedoff /# TheexampleofListing11-7wasrunonanembeddedboardconfiguredforNFSrootmount.We exportadirectoryonourworkstationthatcontainsthesimplefilesystemimagedetailedinListing 11-6.Asoneofthefinalstepsinthebootprocess,theLinuxkernelonourtargetboardmountsaroot file system via NFS. When the kernel attempts to execute /sbin/init, it fails because there is no /sbin/initonourfilesystem.However,aswehaveseen,thekernelalsoattemptstoexecute/bin/sh.In ourBusyBox-configuredtarget,thissucceeds,andbusyboxislaunchedviathesymlink/bin/shonour rootfilesystem. The first thing BusyBox displays is the complaint that it can't find /etc/init.d/rcS. This is the default initialization script that BusyBox searches for. Instead of using inittab, this is the preferred methodtoinitializeanembeddedsystembasedonBusyBox. Whenithascompletedinitialization,BusyBoxdisplaysapromptaskingtheusertopressEnterto activate a console. When it detects the Enter key, it executes an ash shell session waiting for user input. The final message about job control is a result of the fact that we are creating the system consoleonaserialterminal.TheLinuxkernelcontainscodetodisablejobcontrolifitdetectsthe consoleonaserialterminal. This example produced a working system, with nearly 100 Linux utilities available, including core utilities, file utilities, network support, and a reasonably capable shell. You can see that this simplepackageprovidesapowerfulplatformuponwhichtobuildyourownsystemapplications.Of course,itshouldbenotedthatwithoutanysupportforlibcandothersystemlibraries,youwouldface a formidable task implementing your applications. You would have to provide support for all the usualsystemcallsandotherlibraryfunctionsthatatypicalCprogramrelieson.Alternatively,you couldstaticallylinkyourapplicationsagainstthelibrariesitdependson,butifyouhavemorethana couple applications using this method, your applications will likely exceed the combined size of linkingdynamicallyandhavingthesharedlibrariesonyourtarget. 11.3.2.ExamplercSInitializationScript Before BusyBoxspawnsaninteractiveshell,ittriestoexecutecommandsfroma scriptcalled /etc/init.d/rcS,asshowninListing11-7.ItisherewhereyourapplicationscometolifeinaBusyBox system.AsimplercSinitializationscriptisprovidedinListing11-8. Listing11-8.SimplercSBusyBoxStartupScript #!/bin/sh echo"Mountingproc" mount-tproc/proc/proc echo"Startingsystemloggers" syslogd klogd echo"Configuringloopbackinterface" ifconfiglo127.0.0.1 echo"Startinginetd" xinetd #startashell busyboxsh Thissimplescriptismostlyself-explanatory.First,itisimportanttomountthe/procfilesystem onitsreservedmountpoint,/proc.Thisisbecausemanyutilitiesgettheirinformationfromthe/proc filesystem.ThisisexplainedmorefullyinChapter9.Nextwelaunchthesystemloggersasearlyas possible,tocaptureanystartupproblems.Followingthesystemlogdaemons,weconfigurethelocal loopback interface for the system. Again, a number of traditional Linux facilities assume that a loopback interface is present, and if your system has support for sockets configured, you should enable this pseudo interface. The last thing we do before starting a shell is launch the Internet superserver xinetd. This program sits in the background listening for network requests on any configurednetworkinterfaces.Forexample,toinitiateatelnetsessiontotheboard,xinetdintercepts therequestfortelnetconnectionandspawnsatelnetservertohandlethesession. Instead of starting a shell, your own applications can be launched from this rcS initialization script.Listing11-8isasimpleexampleofaTelnet-enabledtargetboardrunningbasicservicessuch assystemandkernelloggers. 11.3.3.BusyBoxTargetInstallation The discussion of BusyBox installation can proceed only when you understand the use and purposeofsymlinks.TheBusyBoxmakefilecontainsatargetcalledinstall.Executingmakeinstall createsadirectorystructurecontainingthebusyboxexecutableandasymlinktree.Thisenvironment needstobemigratedtoyourtargetembeddedsystem'srootdirectory,completewiththesymlinktree. Thesymlinktreeeliminatestheneedtotypebusyboxcommandforeachcommand.Instead,toseea listing of files in a given directory, the user need only type ls. The symlink executes busybox as describedpreviouslyandinvokesthelsfunctionality.ReviewListing11-4andListing11-5toseethe symlinktree.NotethattheBusyBoxbuildsystemcreateslinksonlyforthefunctionalitythatyouhave enabledviatheconfigurationutility. The easiest waytopopulateyourrootfilesystemwiththenecessarysymlinkfarm is to letthe BusyBox build system do it for you. Simply mount your root file system on your development workstationandpassaPREFIXtotheBusyBoxmakefile.Listing11-9showstheprocedure. Listing11-9.InstallingBusyBoxonRootFileSystem $mount-oloopbbrootfs.ext2/mnt/remote $makePREFIX=/mnt/remoteinstall /bin/shapplets/install.sh/mnt/remote /mnt/remote/bin/ash->busybox /mnt/remote/bin/cat->busybox /mnt/remote/bin/chgrp->busybox /mnt/remote/bin/chmod->busybox /mnt/remote/bin/chown->busybox ... /mnt/remote/usr/bin/xargs->../../bin/busybox /mnt/remote/usr/bin/yes->../../bin/busybox /mnt/remote/usr/sbin/chroot->../../bin/busybox -------------------------------------------------Youwillprobablyneedtomakeyourbusyboxbinary setuidroottoensureallconfiguredappletswill workproperly. -------------------------------------------------$chmod+s/mnt/remote/bin/busybox $ls-l/mnt/remote/bin/busybox -rwsr-sr-x1rootroot863188Dec415:54/mnt/remote/bin/busybox First we mount the root file system binary image on our desired mount pointin this case, /mnt/remote, a favorite of mine. Then we invoke the BusyBox make install command, passing it a PREFIXspecifyingwherewewantthesymlinktreeandbusyboxexecutablefiletobeplaced.Asyou canseefromthelisting,themakefileinvokesascriptcalledapplets/install.shtodothebulkofthe work. The script walks through a file containing all the enabled BusyBox applets and creates a symlinkforeachoneonthepathwehavespecifiedusingthePREFIX.Thescriptisverychatty;it outputs a line for each symlink created. For brevity, only the first few and last few symlink announcementsaredisplayed.Theellipsisinthelistingrepresentsthosewehaveeliminated. The message about setuid is also displayed by the install script, to remind you that it might be necessary to make your busybox executable setuid root. This is to allow BusyBox functions that require root access to function properly even when invoked by a nonroot user. This is not strictly necessary, especially in an embedded Linux environment, where it is common to have only a root accountonasystem.Ifthisisnecessaryforyourinstallation,therequiredcommand(chmod+s)is showninListing11-9. Theresultofthisinstallationstepisthatthebusyboxbinaryandsymlinktreeareinstalledonour targetrootfilesystem.TheendresultlooksverysimilartoListing11-4. ItisusefultonotethatBusyBoxalsohasanoptiontoenablecreationofthissymlinktreeonthe target system at runtime. This option is enabled in the BusyBox configuration and is invoked at runtimebyexecutingbusyboxwiththe-installoption.Youmusthavethe/procfilesystemmountedon yourtargetsystemforthissupporttowork. 11.3.4.BusyBoxCommands InarecentBusyBoxsnapshot,197commands(alsocalledapplets)weredocumentedintheman page. There is sufficient support for reasonably complex shell scripts, including support for Bash shell scripting. BusyBox has support for awk and sed, frequently found in Bash scripts. BusyBox supports network utilities such as ping, ifconfig, TRaceroute, and netstat. Some commands are specificallyincludedforscriptingsupport,includingtrue,false,andyes. Spend a few moments perusing Appendix C, "BusyBox Commands," where you can find a summaryofeachBusyBoxcommand.Afteryouhavedoneso,youwillhaveabetterappreciationfor thecapabilitiesofBusyBoxandhowitmightbeapplicabletoyourownembeddedLinuxproject. Asmentionedatthebeginningofthischapter,manyoftheBusyBoxcommandscontainalimited subset of features and options compared to their full-featured counterparts. In general, you can get help on any given BusyBox command at runtime by invoking the command with the --help option. This produces a usage message with a brief description of each supported command option. The BusyBoxgzipappletisausefulexampleofaBusyBoxcommandthathassupportforalimitedsetof options.Listing11-10displaystheoutputfromgzip-helponaBusyBoxtarget. Listing11-10.BusyBoxgzipAppletUsage /#gzip--help BusyBoxv1.01(2005.12.01-21:11+0000)multi-callbinary Usage:gzip[OPTION]...[FILE]... CompressFILE(s)withmaximumcompression. WhenFILEis'-'orunspecified,readsstandardinput.Implies-c. Options: -cWriteoutputtostandardoutputinsteadofFILE.gz -dDecompress -fForcewritewhendestinationisaterminal The BusyBox version of gzip supports just three command line options. Its full-featured counterpartcontainssupportformorethan15differentcommandlineoptions.Forexample,thefullfeaturedgziputilitysupportsa--listoptionthatproducescompressionstatisticsforeachfileonthe commandline.NosuchsupportexistsforBusyBoxgzip.Thisisusuallynotasignificantlimitation for embedded systems. We present this information so you can make an informed choice when decidingonBusyBox.Whenthefullcapabilitiesofautilityareneeded,thesolutionissimple:Delete supportforthatparticularutilityintheBusyBoxconfigurationandaddthestandardLinuxutilityto yourtargetsystem. 11.4.ChapterSummary •BusyBoxisapowerfultoolforembeddedsystemsthatreplacesmanycommonLinuxutilitiesin asinglemulticallbinary. •BusyBoxcansignificantlyreducethesizeofyourrootfilesystemimage. •BusyBoxiseasytouseandhasmanyusefulfeatures. • Configuring BusyBox is straightforward, using an interface similar to that used for Linux configuration. •BusyBoxcanbeconfiguredasastaticallyordynamicallylinkedapplication,dependingonyour particularrequirements. • System initialization is somewhat different with BusyBox; those differences were covered in thischapter. • BusyBox has support for many commands. Appendix C itemizes all the available BusyBox commandsfromarecentrelease. 11.4.1.SuggestionsforAdditionalReading BusyBoxProjecthome www.busybox.net/ BusyBoxmanpage www.busybox.net/downloads/BusyBox.html Chapter12.EmbeddedDevelopmentEnvironment Theconfigurationandservicesenabledonyourhostdevelopmentsystemcanhaveahugeimpact on your success as an embedded developer. This chapter examines the unique requirements of a cross-development environment and some of the tools and techniques that an embedded developer needstoknowtobeproductive. We begin by examining a typical cross-development environment. Using the familiar "hello world" example, we detail the important differences between host-based applications and those targeted at embedded systems. We also look at differences in the toolchains for native versus embeddedapplicationdevelopment.Wethenpresenthostsystemrequirementsanddetailtheuseof someimportantelementsofyourhostsystem.Weconcludethischapterwithanexampleofatarget boardbeinghostedbyanetwork-basedhost. 12.1.Cross-DevelopmentEnvironment Developers new to embedded development often struggle with the concepts and differences between native and cross-development environments. Indeed, there are often three compilers and three(ormore)versionsofstandardheaderfilessuchasstdlib.h.Debugginganapplicationonyour target embedded system can be difficult without the right tools and host-based utilities. You must manageandseparatethefilesandutilitiesdesignedtorunonyourhostsystemfromthoseyouintend touseonyourtarget. Whenweusethetermhostinthiscontext,wearereferringtothedevelopmentworkstationthatis sittingonyourdesktopandrunningyourfavoriteLinuxdesktopdistribution.[79]Conversely,whenwe use the term target we are referring to your embedded hardware platform. Therefore, native development denotes the compilation and building of applications on and for your host system. Cross-developmentdenotesthecompilationandbuildingofapplicationsonthehostsystemthatwill berunontheembeddedsystem.Keepingthesedefinitionsinmindwillhelpyoustayontrackthrough thischapter. Figure 12-1 shows the layout of a typical cross-development environment. A host PC is connectedtoatargetboardviaoneormorephysicalconnections.Itismostconvenientifbothserial and Ethernet ports are available on the target. Later when we discuss kernel debugging, you will realizethatasecondserialportcanbeaveryvaluableasset. Figure12-1.Cross-developmentsetup Inthemostcommonscenario,thedeveloperhasaserialterminalonthehostconnectedtotheRS232serialport,possiblyoneormoreTelnetterminalsessionstothetargetboard,andpotentiallyone or more debug sessions using Ethernet as the connection medium. This cross-development setup providesagreatdealofflexibility.Thebasicideaisthatthehostsystemprovidesthehorsepowerto run the compilers, debuggers, editors, and other utilities, while the target executes only the applicationsdesignedforit.Yes,youcancertainlyruncompilersanddebuggersonthetargetsystem, but we assume that your host system contains more resources, including RAM, disk storage, and Internetconnectivity.Infact,itisnotuncommonforatargetembeddedboardtohavenohuman-input devicesoroutputdisplays. 12.1.1."HelloWorld"Embedded A properly configured cross-development system hides a great deal of complexity from the averageapplicationdeveloper.Lookingatasimpleexamplewillhelpuncoverandexplainsomeof themystery.Whenwecompileasimple"helloworld"program,thetoolchain(compiler,linker,and associated utilities) makes many assumptions about the host system we are building on and the program we are compiling. Actually, they are not assumptions, but a collection of rules that the compilerreferencestobuildaproperbinary. Listing12-1reproducesasimple"helloworld"program. Listing12-1.HelloWorldAgain #include<stdio.h> intmain(intargc,char**argv){ printf("HelloWorld\n"); return0; } Even the casual application developer will realize some important points about this C source file. First, the function printf() is referenced but not defined in this file. If we omit the #include directivecontainingtheprototypefortheprintf()function,thecompileremitsthefamiliarmessage: hello.c:5:warning:implicitdeclarationoffunction'printf' Thisintroducessomeinterestingquestions: •Whereisthefilestdio.hlocated,andhowisitfound? • Where does the printf() function live, and how is this reference resolved in the binary executable? Somehowitseemsthatthecompilerjustknowshowtoputtogetheraproperbinaryfilethatis executable from the command line. To further complicate matters, the final executable contains startupandshutdownprologuecodethatweneverseebutthatthelinkerautomaticallyincludes.This prologuedealswithdetailssuchastheenvironmentandargumentspassedtoyourprogram,startup andshutdownhousekeeping,exithandling,andmore. To build the "hello world" application, we can use a simple command line invocation of the compiler,similartothis: $gcc-ohellohello.c This produces the binary executable file called hello, which we can execute directly from the commandline.Defaultsreferencedbythecompilerprovideguidanceonwhereincludefileswillbe found.Inasimilarfashion,thelinkerknowshowtoresolvethereferencetotheprintf()functionby includingareferencetothelibrarywhereitisdefined.This,ofcourse,isthestandardClibrary. Wecanquerythetoolchaintoseesomeofthedefaultsthatwereused.Listing12-2isapartial listing of the output from cpp when passed the -v flag. You might already know that cpp is the C preprocessorcomponentofthegcctoolchain.Wehaveaddedsomeformatting(whitespaceonly)to improvethereadability. Listing12-2.DefaultNativecppSearchDirectories $cpp-v Readingspecsfrom/usr/lib/gcc-lib/i386-redhat-linux/3.3.3/specs Configuredwith:../configure--prefix=/usr--mandir=/usr/share/man--infodir=/usr/share/info-enable-shared --enable-threads=posix --disable-checking --disable-libunwind-exceptions --withsystem-zlib--enable-__cxa_atexit--host=i386-redhat-linux Threadmodel:posix gccversion3.3.320040412(RedHatLinux3.3.3-7) /usr/lib/gcc-lib/i386-redhat-linux/3.3.3/cc1-E-quiet-vignoringnonexistentdirectory"/usr/i386-redhat-linux/include" #include"..."searchstartshere: #include<...>searchstartshere: /usr/local/include /usr/lib/gcc-lib/i386-redhat-linux/3.3.3/include /usr/include Endofsearchlist. /usr/lib/ This simple query produces some very useful information. First, we can see how the compiler was configured using the familiar ./configure utility. The default thread model is posix, which determinesthethreadlibraryyourapplicationgetslinkedagainstifyouemploythreadingfunctions. Finally,youseethedefaultsearchdirectoriesfor#includedirectives. But what if we want to build hello.c for a different architecture, such as PowerPC? When we compileanapplicationprogramforaPowerPCtargetusingacross-compileronourhostmachine, wemustmakesurethatthecompilerdoesnotusethedefaulthostincludedirectoriesorlibrarypaths. Using a properly configured cross-compiler is the first step, and having a well designed crossdevelopmentenvironmentisthesecond. Listing12-3istheoutputfromapopularopen-sourcecross-developmenttoolchainknownasthe Embedded Linux Development Kit (ELDK), assembled and maintained by Denx Software Engineering.ThisparticularinstallationwasconfiguredforthePowerPC82xxtoolchain.Again,we haveaddedsomewhitespacetotheoutputforreadability. Listing12-3.DefaultCross-SearchDirectories $ppc_82xx-cpp-v Readingspecsfrom/opt/eldk/usr/bin/../lib/gcc-lib/ppc-linux/3.3.3/specs Configuredwith:../configure--prefix=/usr--mandir=/usr/share/man--infodir=/usr/share/info-enable-shared--enable-threads=posix--disable-checking--with-system-zlib --enable-__cxa_atexit --with-newlib --enable-languages=c,c++ --disable-libgcj --host=i386redhat-linux-target=ppc-linux Threadmodel:posix gccversion3.3.3(DENXELDK3.1.13.3.3-10) /opt/eldk/usr/bin/../lib/gcc-lib/ppc-linux/3.3.3/cc1 -E -quiet -v -iprefix /opt/eldk/usr/bin/.. /lib/gcc-lib/ppc-linux/3.3.3/ -D__unix__ -D__gnu_linux__ -D__linux__ -Dunix -D__unix -Dlinux D__linux-Asystem=unix-Asystem=posix--mcpu=603 ignoringnonexistentdirectory"/opt/eldk/usr/ppc-linux/sys-include" ignoringnonexistentdirectory"/opt/eldk/usr/ppc-linux/include" #include"..."searchstartshere: #include<...>searchstartshere: /opt/eldk/usr/lib/gcc-lib/ppc-linux/3.3.3/include /opt/eldk/ppc_82xx/usr/include Endofsearchlist. Hereyoucanseethatthedefaultsearchpathsforincludedirectoriesarenowadjustedtopointto yourcrossversionsinsteadofthenativeincludedirectories.Thisseeminglyobscuredetailiscritical tobeingabletodevelopapplicationsandcompileopen-sourcepackagesforyourembeddedsystem. It is one of the most confusing topics to even experienced application developers who are new to embeddedsystems. 12.2.HostSystemRequirements Your development workstation must include several important components and systems. Of course,youneedaproperlyconfiguredcrosstoolchain.Youcandownloadandcompileoneyourself or obtain one of the many commercial toolchains available. Building one yourself is beyond the scope of this book, although there are several good references available. See Section 12.4.1, "SuggestionsforAdditionalReading,"attheendofthischapterforrecommendations. The next major item you need is a Linux distribution targeted for your embedded system architecture. This includes hundreds to potentially thousands of files that will populate your embedded system's file systems. Again, the choices are to build your own or to obtain one of the commercialones.OneofthemorepopularembeddedsystemdistributionsavailableontheInternetis theaforementionedELDK.TheELDKisavailableforsomePowerPCandotherembeddedtargets. BuildinganembeddedLinuxdistributionfromscratchwouldrequireabookofthissizeinitselfand, therefore,isbeyondthescopeofourdiscussionshere. Insummary,yourdevelopmenthostrequiresfourseparateanddistinctcapabilities: •Crosstoolchainandlibraries •Targetsystempackages,includingprograms,utilities,andlibraries •Hosttoolssuchaseditors,debuggers,andutilities •Serversforhostingyourtargetboard,coveredinthenextsection Ifyouinstallaready-builtembeddedLinuxdevelopmentenvironmentonyourworkstation,either a commercial variety or one freely available in the open source community, the toolchain and componentshavealreadybeenpreconfiguredtoworktogether.Forexample,thetoolchainhasbeen configuredwithdefaultdirectorysearchpathsthatmatchthelocationofthetargetheaderfilesand systemlibrariesonyourdevelopmentworkstation.Thesituationbecomesmuchmorecomplexifyour requirementsincludehavingsupportformultiplearchitecturesandprocessorsonyourdevelopment workstation.ThisisthereasonthatembeddedLinuxdistributionsexist. 12.2.1.HardwareDebugProbe In addition to the components listed previously, you should consider some type of hardwareassisteddebugging.Thisconsistsofahardwareprobeconnectedtoyourhost(oftenviaEthernet)and connectedtoyourtargetviaadebugconnectorontheboard.Manysolutionsareonthemarkettoday. WecoverthistopicindetailinChapter14,"KernelDebuggingTechniques." 12.3.HostingTargetBoards ReferringbacktoFigure12-1,youwillnoticeanEthernetconnectionfromthetarget-embedded board to the host-development system. This is not strictly necessary, and, indeed, some smaller embeddeddevicesdonothaveanEthernetinterface.However,thisistheexceptionratherthanthe rule.HavinganEthernetconnectionavailableonyourtargetboardisworthitscostinsilicon! While developing thekernel, youwillcompileanddownloadkernelstoyourembedded board many times. Many embedded development systems and bootloaders have support for TFTP and assume that the developer will use it. TFTP is a lightweight protocol for moving files between a TFTPserverandTFTPclient,similartoFTP. Using TFTP from yourbootloadertoloadthekernelwillsaveyoucountlesshourswaitingfor serialdownloadsevenathigherserialbaudrates.Andloadingyourramdiskcantakemuchlonger because ramdisk images can grow to many tens of megabytes and more, depending on your requirements.TheinvestmentinyourtimetoconfigureanduseTFTPwillsurelypayoffandishighly recommended.Thereareveryfewdesignsthatcan'taffordtherealestatetoincludeanEthernetport duringdevelopment,evenifitisdepopulatedforproduction. 12.3.1.TFTPServer Configuring TFTP on your Linux development host is not difficult. Of course, the details might vary, depending on which Linux distribution you choose for your development workstation. The guidelinespresentedherearebasedonRedHatandFedoraCoreLinuxdistributions. TFTPisaTCP/IPservicethatmustbeenabledonyourworkstation.ToenableTFTPservice,you must instruct your server to respond to incoming TFTP packets and spawn your TFTP server. On many Linux distributions, this is done by editing a configuration file used by the xinetd Internet superserver. For example, on the Red Hat and Fedora desktop Linux distributions, this file is /etc/xinetd.d/tftp. Listing 12-4 contains a TFTP configuration from a Fedora Core 2 development workstationtoenabletheTFTPservice.Ithasbeenslightlyrearrangedtofitthepage. Listing12-4.TFTPConfiguration #default:off #description:Thetftpserverservesfilesusingthetrivial #filetransferprotocol.Thetftpprotocolisoftenusedto #bootdisklessworkstations,downloadconfigurationfilesto #network-awareprinters,andtostarttheinstallationprocess #forsomeoperatingsystems. servicetftp { socket_type=dgram protocol=udp wait=yes user=root server=/usr/sbin/in.tftpd server_args=-c-s/tftpboot disable=no per_source=11 cps=1002 flags=IPv4 } Inthistypicalsetup,theTFTPservicehasbeenenabled(disable=no)andconfiguredtoserve fileslocatedinthisworkstation's/tftpbootdirectory.WhenthexinetdInternetsuperserverreceives an incoming TFTP request, it consults this configuration and spawns the server specified (/usr/sbin/in.tftpd).Thecommandlineargumentsspecifiedbyserver_argsarepassedtothein.tftpd process.Inthiscase,the-sswitchtellsin.tftpdtoswitchtothespecifieddirectory(/tftpboot),and the-cflagallowsthecreationofnewfiles.Thisisusefultowritefilestotheserverfromthetarget. Consult the documentation that came with your desktop distribution for details specific to your environment. 12.3.2.BOOTP/DHCPServer Having a DHCP server on your development host simplifies the configuration management for your embedded target. We have already established the reasons why an Ethernet interface on your target hardware is a good idea. When Linux boots on your target board, it needs to configure the Ethernetinterfacebeforetheinterfacewillbeuseful.Moreover,ifyouareusinganNFSrootmount configurationonyourtargetboard,Linuxneedstoconfigureyourtarget'sEthernetinterfacebeforethe bootprocesscancomplete.WecoveredNFSindetailinChapter9,"FileSystems." Ingeneral,LinuxcanusetwomethodstoinitializeitsEthernet/IPinterfaceduringboot: •Hard-codetheEthernetinterfaceparameterseitherontheLinuxkernelcommandlineorinthe defaultconfiguration •Configurethekerneltoautomaticallydetectthenetworksettingsatboottime Forobviousreasons,thelatterchoiceisthemostflexible.DHCPorBOOTPistheprotocolyour target and server use to accomplish the automatic detection of network settings. For details of the DHCPorBOOTPprotocols,seeSection12.4.1attheendofthischapter. A DHCP server controls the IP address assignments for IP subnets for which it has been configured, and for DHCP or BOOTP clients that have been configured to participate. A DHCP serverlistensforrequestsfromaDHCPclient(suchasyourtargetboard),andassignsaddressesand otherpertinentinformationtotheclientaspartofthebootprocess.AtypicalDHCPexchange(see Listing12-5)canbeexaminedbystartingyourDHCPserverwiththe-ddebugswitchandobserving theoutputwhenatargetmachinerequestsconfiguration. Listing12-5.TypicalDHCPExchange tgt>DHCPDISCOVERfrom00:09:5b:65:1d:d5viaeth0 svr>DHCPOFFERon192.168.0.9to00:09:5b:65:1d:d5viaeth0 tgt>DHCPREQUESTfor192.168.0.9(192.168.0.1)from\ 00:09:5b:65:1d:d5viaeth0 svr>DHCPACKon192.168.0.9to00:09:5b:65:1d:d5viaeth0 Thesequencestartswiththeclient(target)transmittingabroadcastframeattemptingtodiscover aDHCPserver.ThisisshownbytheDHCPDISCOVERmessageshown.Theserverresponds(ifit hasbeensoconfiguredandenabled)byofferinganIPaddressfortheclient.Thisisevidencedbythe DHCPOFFER message. The client then responds by testing this IP address locally. The testing includes sending the DHCPREQUEST packet to the DHCP server, as shown. Finally, the server responds by acknowledging the IP address assignment to the client, thus completing the automatic targetconfiguration. It is interesting to note that a properly configured client will remember the last address it was assigned by a DHCP server. The next time it boots, it will skip the DHCPDISCOVER stage and proceeddirectlytotheDHCPREQUESTstage,assumingthatitcanreusethesameIPaddressthatthe serverpreviouslyassigned.AbootingLinuxkerneldoesnothavethiscapabilityandemitsthesame sequenceeverytimeitboots. Configurationofyourhost'sDHCPserverisnotdifficult.Asusual,ouradviceistoconsultthe documentation that came with your desktop Linux distribution. On a Red Hat or Fedora Core distribution,theconfigurationentryforasingletargetmightlooklikeListing12-6. Listing12-6.ExampleDHCPServerConfiguration #ExampleDHCPServerconfiguration allowbootp; subnet192.168.1.0netmask255.255.255.0{ default-lease-time1209600;#twoweeks optionrouters192.168.1.1; optiondomain-name-servers1.2.3.4; group{ hostpdna1{ hardwareethernet00:30:bd:2a:26:1f; fixed-address192.168.1.68; filename"uImage-pdna"; optionroot-path"/home/chris/sandbox/pdna-target"; } } } Thisisasimpleexample,meantonlytoshowthekindofinformationyoucanpasstoyourtarget system. There is a one-to-one mapping of the target MAC address to its assigned IP address. In addition to its fixed IP address, you can pass other information to your target. In this example, the defaultrouterandDNSserveraddressesarepassedtoyourtarget,alongwiththefilenameofafile ofyourchoice,andarootpathforyourkerneltomountanNFSrootmountfrom.Thefilenamemight beusedbyyourbootloadertoloadakernelimagefromyourTFTPserver.Youcanalsoconfigure yourDHCPservertohandoutIPaddressesfromapredefinedrange,butitisveryconvenienttousea fixedaddresssuchasthatshowninListing12-6. YoumustenabletheDHCPserveronyourLinuxdevelopmentworkstation.Thisistypicallydone through your main menu or via the command line. Consult the documentation for your own Linux distributionfordetailssuitableforyourenvironment.Forexample,toenabletheDHCPserverona FedoraCore2Linuxdistribution,simplytypethefollowingcommandfromarootcommandprompt: $/etc/init.d/dhcpdstart(orrestart) You must do this each time you start your development workstation, unless you configure it to startautomatically. ManynuancesareinvolvedwithinstallingaDHCPserver,sounlessyourserverisonaprivate network,itisadvisabletocheckwithyoursystemadministratorbeforegoinglivewithyourown.If you coexist with a corporate LAN, it is very possible that you will interfere with its own DHCP service. 12.3.3.NFSServer UsinganNFSrootmountforyourtargetboardisaverypowerfuldevelopmenttool.Someofthe advantagesofthisconfigurationfordevelopmentare: •Yourrootfilesystemisnotsize-restrictedbyyourboard'sownlimitedresources,suchasFlash memory. •Changesmadetoyourapplicationfilesduringdevelopmentareimmediatelyavailabletoyour targetsystem. •Youcandebugandbootyourkernelbeforedevelopinganddebuggingyourrootfilesystem. SettingupanNFSservervariesdependingonthedesktopLinuxdistributionyouareusing.As withtheotherservicesdescribedinthischapter,youmustconsultthedocumentationforyourown Linuxdistributionforthedetailsappropriatetoyourconfiguration.TheNFSservicemustbestarted fromeitheryourstartupscripts,agraphicalmenu,orthecommandline.Forexample,thecommandto startNFSservicesfromarootcommandpromptforaFedoraCore2Linuxdesktopisasfollows: $/etc/init.d/nfsstart(orrestart) YoumustdothiseachtimeyoustartyourdesktopLinuxworkstation.(Thisandotherservicescan bestartedautomaticallyonbootingconsultthedocumentationforyourdesktopLinuxdistribution.)In addition to enabling the service, your kernel must be compiled with support for NFS. Although DHCPandTFTParebothuserspaceutilities,NFSrequireskernelsupport.Thisistrueonbothyour developmentworkstationandyourtargetboard.Figure12-2illustratestheconfigurationoptionsfor NFSinthekernel.NoticethatthereareconfigurationoptionsforbothNFSserverandclientsupport. NotealsotheoptionforrootfilesystemonNFS.Yourtargetkernelmusthavethisoptionconfigured forNFSrootmountoperation. Figure12-2.NFSkernelconfiguration TheNFSservergetsitsinstructionsfromanexportsfilelocatedonyourserver.Itiscommonly foundin/etc/exports.Listing12-7isanexampleofasimpleexportsentry. Listing12-7.SimpleNFSexportsFile $cat/etc/exports #/etc/exports /home/chris/sandbox/coyote-target*(rw,sync,no_root_squash) /home/chris/sandbox/pdna-target*(rw,sync,no_root_squash) /home/chris/workspace*(rw,sync,no_root_squash) These entries on my workstation allow a client to remotely mount any of the three directories shown. The attributes following the directory specification instruct the NFS server to allow connectionsfromanyIPaddress(*)andtomounttherespectivedirectorieswiththegivenattributes (read/write with no_root_squash). The latter attribute enables a client with root privileges to exercisethoseprivilegesonthegivendirectory.Itisusuallyrequiredwhenworkingwithembedded systemsbecausetheyoftenhaveonlyrootaccounts. YoucantestyourNFS configurationrightfromyourworkstation.Assuming thatyouhaveNFS servicesenabled(requiresbothNFSserverandclientcomponentsenabled),youcanmountalocal NFSexportasyouwouldmountanyotherfilesystem: #mount-tnfslocalhost:/home/chris/workspace/mnt/remote Ifthiscommandsucceedsandthefilesin.../workspaceareavailableon/mnt/remote,yourNFS serverconfigurationisworking. 12.3.4.TargetNFSRootMount MountingyourtargetviaNFSrootmountisnotdifficult,and,asmentionedelsewhere,itisavery usefuldevelopmentconfiguration.However,asetofdetailsmustbecorrectbeforeitwillwork.The stepsrequiredareasfollows: 1.ConfigureyourNFSserverandexportapropertargetfilesystemforyourarchitecture. 2.ConfigureyourtargetkernelwithNFSclientservicesandrootfilesystemonNFS. 3.Enablekernel-levelautoconfigurationofyourtarget'sEthernetinterface. 4. Provide your target Ethernet IP configuration via the kernel command line or static kernel configurationoption. 5.ProvideakernelcommandlineenabledforNFS. We presented the kernel configuration in Figure 12-2 when we explained the NFS server configuration. You must make sure that your target kernel configuration has NFS client services enabled, and, in particular, you must enable the option for Root file system on NFS. Specifically, make sure that your kernel has CONFIG_NFS_FS=y and CONFIG_ROOT_NFS=y. Obviously, you cannotconfigureNFSasloadablemodulesifyouintendtobootNFSrootmount. Kernel-levelautoconfigurationisaTCP/IPconfigurationoptionfoundundertheNetworkingtab inthekernelconfigurationutility.EnableCONFIG_IP_PNPonyourtargetkernel.Whenselected,you are presented with several options for automatic configuration. Select either BOOTP or DHCP, as describedearlier.Figure12-3illustratesthekernelconfigurationforkernel-levelautoconfiguration. Figure12-3.Kernel-levelautoconfiguration When your server and target kernel are configured, you need to provide your target Ethernet configurationviaoneofthemethodsdescribedearlier.Ifyourbootloadersupportsakernelcommand line,thatistheeasiestmethod.HereiswhatakernelcommandlinemightlookliketosupportNFS rootmount: console=ttyS0,115200root=/dev/nfsrwip=dhcp\ nfsroot=192.168.1.9:/home/chris/sandbox/pdna-target 12.3.5.U-BootNFSRootMountExample U-Boot is a good example of a bootloader that supports a configurable kernel command line. Using U-Boot's nonvolatile environment feature, we can store our kernel command line in a parameterspeciallynamedforthispurpose.ToenabletheNFScommandlineinU-Boot,wedothe following(allononelineinourserialterminal): setenvbootargsconsole=ttyS0,115200root=/dev/nfsrw\ ip=dhcpnfsroot=192.168.1.9:/home/chris/sandbox/pdna-target ThenweloadakernelviaourTFTPserver.Listing12-8showswhatthismightlooklikeona PowerPCembeddedtarget. Listing12-8.LoadingKernelviaTFTPServer =>tftpboot200000uImage-pdna<<<EnteredatU-Bootprompt UsingFECETHERNETdevice TFTPfromserver192.168.1.9;ourIPaddressis192.168.1.68 Filename'uImage-pdna'. Loadaddress:0x200000 Loading:################################################## ################################################## ######################################### done Bytestransferred=911984(dea70hex) => Whenwebootthekernel,weseespecificevidenceofourNFSrootmountconfiguration.Listing 12-9reproducesselectedoutputfromthekernelbootmessagestodemonstratethis.Thisoutputhas beenformatted(manylinesomittedandwhitespaceadded)forreadability. Listing12-9.BootingwithNFSRootMount UncompressingKernelImage...OK Linuxversion2.6.14(chris@pluto)(gccversion3.3.3(DENXELDK3.1.13.3.3-10))#1Mon Jan211:58:48EST2006 . . Kernel command line: console=ttyS0,115200 root=/dev/nfs rw nfsroot=192.168.1.9:/home /chris/sandbox/pdna-targetip=dhcp . . SendingDHCPrequests...OK IP-Config:GotDHCPanswerfrom192.168.1.9,myaddressis192.168.1.68 IP-Config:Complete: device=eth0,addr=192.168.1.68,mask=255.255.255.0, gw=255.255.255.255,host=192.168.1.68,domain=, nis-domain=(none),bootserver=192.168.1.9, rootserver=192.168.1.9, rootpath=/home/chris/sandbox/pdna-target . . LookingupportofRPC100003/2on192.168.1.9 LookingupportofRPC100005/1on192.168.1.9 VFS:Mountedroot(nfsfilesystem). . . BusyBoxv0.60.5(2005.06.07-07:03+0000)Built-inshell(msh) Enter'help'foralistofbuilt-incommands. # From Listing 12-9, first we see the kernel banner followed by the kernel command line. We specifiedfouritemsinthiskernelcommandline: •Consoledevice(/dev/console) •Rootdevice(/dev/nfs) •NFSrootpath(/home/chris/sandbox/pdna-target) •IPkernel-levelautoconfigurationmethod(dhcp) Shortlythereafter,weseethekernelattemptingkernel-levelautoconfigurationviaDHCP.When theserverrespondsandtheDHCPexchangecompletes,thekerneldisplaysthedetectedconfiguration inthefollowinglines.YoucanseefromthislistingthattheDHCPserverhasassignedthetargetthe IP address 192.168.1.68. Compare the detected settings with those specified in Listing 12-6. That wassimilartotheDHCPserverconfigurationthatresultedinthisconfiguration. When the kernel has completed the IP autoconfiguration, it is capable of mounting the root file system using the supplied parameters. You can see this from the three lines ending with the VFS (virtualfilesubsystem)messageannouncingthatithasmountedtherootNFSfilesystem.Afterthe NFSrootfilesystemhasbeenmounted,initializationcompletesasdescribedinChapter5,"Kernel Initialization." ItisalsopossibletopasstargetIPsettingstothekernelinastaticfashioninsteadofhavingthe kernel obtain IP settings from a DHCP or BOOTP server. IP settings can be passed via the kernel commandlinedirectly.Inthiscase,thekernelcommandlinemightlooksimilartothis: console=console=ttyS0,115200\ip=192.168.1.68:192.168.1.9::255.255.255.0:pdna:eth0:off\ root=/dev/nfsrwnfsroot=192.168.1.9:/home/chris/pdna-target 12.4.ChapterSummary •Manyfeaturesofadevelopmentenvironmentgreatlyfacilitateefficiencyforembeddedcrossdevelopment. Most of these fall under the category of tools and utilities. We cover this aspect in detailinthenextchapter,wherewecoverdevelopmenttools. •Aproperlyconfigureddevelopmenthostisacriticalassetfortheembeddeddeveloper. • Toolchains employed for cross-development must be properly configured to match your host system'stargetLinuxenvironment. • Your development host must have target components installed that your toolchain and binary utilities can reference. These components include target header files, libraries, target binaries, and their associated configuration files. In short, you need to assemble or obtain an embedded Linux distribution. • Configuring target servers such as TFTP, DHCP, and NFS will greatly increase your productivity as an embedded Linux developer. This chapter introduced configuration examples for each. 12.4.1.SuggestionsforAdditionalReading GCConlinedocumentation http://gcc.gnu.org/onlinedocs/ Buildingandtestinggcc/glibccrosstoolchains http://kegel.com/crosstool/ TheTFTPProtocol,Version2 RFC1350 www.ietf.org/rfc/rfc1350.txt?number=1350 BootstrapProtocol(BOOTP) RFC951 www.ietf.org/rfc/rfc0951.txt?number=951 DynamicHostConfigurationProtocol RFC2131 www.ietf.org/rfc/rfc2131.txt?number=2131 Chapter13.DevelopmentTools AtypicalembeddedLinuxdistributionincludesmanyusefultools.Somearecomplexandrequire agreatdealofproficiencytomaster.Othersaresimpleandhavebeenallbutignoredbydevelopers of embedded systems. Some tools might require customization for a particular environment. Many willrun"rightoutofthebox"andprovidethedeveloperwithusefulinformationwithoutmucheffort. Thischapterpresentsacross-sectionofthemostimportant(andfrequentlyneglected)toolsavailable totheembeddedLinuxengineer. It is impossible to provide complete details on the tools and utilities presented in this chapter. That would take an entire book by itself! Rather than provide a complete reference, our goal is to provide an introduction on the basic usage of each one. You are encouraged to pursue additional study on these and other important development tools. The man page (or other documentation) for eachtoolisagreatplacetostart. The GNU Debugger (GDB) is introduced first, followed by a brief look at the Data Display Debugger,agraphicalfrontendforGDB.Nextweintroduceaseriesofutilitiesdesignedtogivethe developeralookatthebehaviorofprogramsandthesystemasawhole.Theseincludestrace,ltrace, top,andps,oftenoverlookedbyinexperiencedLinuxdevelopers.Wethenpresentsomecrashdump and memory-analysis tools. The chapter concludes by introducing some of the more useful binary utilities. 13.1.GNUDebugger(GDB) IfyouspendmuchtimedevelopingLinuxapplications,youwillundoubtedlyspendmanyhours getting to know the GNU Debugger. GDB is arguably the most important tool in the developer's toolbox. It has a long history, and its capabilities have blossomed to include low-level hardwarespecific debugging support for a wide variety of architectures and microprocessors. It should be notedthattheusermanualforGDBisnearlyaslargeasthisbook.Ourintentionhereistointroduce GDBtogetyoustarted.YouareencouragedtostudytheusermanualreferencedlaterunderSection 13.7.1,"SuggestionsforAdditionalReading." BecausethisisabookaboutembeddedLinuxdevelopment,weuseaversionofGDBthathas beencompiledasacross-debugger.Thatis,thedebuggeritselfrunsonyourdevelopmenthost,butit understandsbinaryexecutablesinthearchitectureforwhichitwasconfiguredatcompiletime.Inthe nextfewexamples,weuseGDBcompiledforaRedHatLinux-compatibledevelopmenthost,andan XScale(ARM)targetprocessor.Althoughweusetheshortnamegdb,wearepresentingexamples basedontheXScale-enabledcross-gdbfromtheMontaVistaembeddedLinuxdistributionforARM XScale. The binary is called xscale_be-gdb. It is still GDB, simply configured for a crossdevelopmentenvironment. The GDB debugger is a complex program with many configuration options during the build process. It is not our intention to provide guidance on building gdb that has been covered in other literature. For the purposes of this chapter, we assume that you have obtained a working GDB configuredforthearchitectureandhostdevelopmentenvironmentyouwillbeusing. 13.1.1.DebuggingaCoreDump OneofthemostcommonreasonstodragGDBoutofthetoolboxistoevaluateacoredump.Itis quickandeasy,andoftenleadstoimmediateidentificationoftheoffendingcode.Acoredumpresults whenanapplicationprogramgeneratesafault,suchasaccessingamemorylocationthatitdoesnot own. Manyconditionscantriggera coredump,[80] but SIGSEGV (segmentation fault) is by far the mostcommon.ASIGSEGVisaLinuxkernelsignalthatisgeneratedonillegalmemoryaccessesby a user process. When this signal is generated, the kernel terminates the process. The kernel then dumpsacoreimage,ifsoenabled. Toenablegenerationofacoredump,yourprocessmusthavetheresourcelimitstoenableacore dump.Thisisachievedbysettingtheprocess'sresourcelimitsusingthesetrlimit()functioncall,or from a BASH or BusyBox shell command prompt, using ulimit. It is not uncommon to find the following line in the initialization scripts of an embedded system to enable the generation of core dumpsonprocesserrors: $ulimit-cunlimited This BASH built-in command is used to set the size limit of a core dump. In the previous instance,thesizeissettounlimited. When an application program generates a segmentation fault (for example, by writing to a memory address outside its permissible range), Linux terminates the process and generates a core dump,ifsoenabled.Thecoredumpisasnapshotoftherunningprocessatthetimethesegmentation faultoccurred. It helps to have debugging symbols enabled in your binary. GDB produces much more useful output with debugging symbols (gcc -g) enabled during the build. However, it is still possible to determinethesequenceofeventsleadingtothesegmentationfault,evenifthebinarywascompiled withoutdebuggingsymbols.Youmightneedtodoabitmoreinvestigativeworkwithouttheaidof debuggingsymbols.Youmustmanuallycorrelatevirtualaddressestolocationswithinyourprogram. Listing13-1showstheresultsofacoredumpanalysissessionusingGDB.Theoutputhasbeen reformatted slightly to fit the page. We have used some demonstration software to intentionally produce a segmentation fault. Here is the output of the process (called webs) that generated the segmentationfault: root@coyote:/workspace/websdemo#./webs Segmentationfault(coredumped) Listing13-1.CoreDumpAnalysisUsingGDB $xscale_be-gdbwebscore GNUgdb6.3(MontaVista6.3-20.0.22.05011312005-07-23) Copyright2004FreeSoftwareFoundation,Inc. GDBisfreesoftware,coveredbytheGNUGeneralPublic License,andyouarewelcometochangeitand/ordistributecopiesofitundercertainconditions. Type"showcopying"toseetheconditions. ThereisabsolutelynowarrantyforGDB.Type"showwarranty"fordetails. This GDB was configured as "--host=i686-pc-linux-gnu -target=armv5teb-montavista- linuxeabi"... Corewasgeneratedby'./webs'. Programterminatedwithsignal11,Segmentationfault. Readingsymbolsfrom/opt/montavista/pro/.../libc.so.6...done. Loadedsymbolsfor/opt/montavista/pro/.../libc.so.6 Readingsymbolsfrom/opt/montavista/pro/.../ld-linux.so.3...done. Loadedsymbolsfor/opt/montavista/pro/.../ld-linux.so.3 #00x00012ac4inClearBlock(RealBigBlockPtr=0x0,l=100000000)atled.c:43 43*ptr=0; (gdb)l 38 39staticintClearBlock(char*BlockPtr,intl) 40{ 41char*ptr; 42for(ptr=BlockPtr;(ptr-BlockPtr)<l;ptr++) 43*ptr=0; 44return0; 45} 46staticintInitBlock(char*ptr,intn) 47{ (gdb)pptr $1=0x0 (gdb) 13.1.2.InvokingGDB ThefirstlineofListing13-1showshowGDBwasinvokedfromthecommandline.Becausewe aredoingcross-debugging,weneedthecross-versionofGDBthathasbeencompiledforourhost andtargetsystem.Weinvokeourversionofcross-gdbasshownandpassxscale_be-gdbthenameof the binary followed by the name of the core dump filein this case, simply core. After GDB prints several banner lines describing its configuration and other information, it prints the reason for the termination:signal11,theindicationofasegmentationfault.[81]SeverallinesfollowasGDBloads thebinary,thelibrariesitdependson,andthecorefile.ThelastlineprinteduponGDBstartupisthe currentlocationoftheprogramwhenthefaultoccurred.Thelineprecededbythe#0stringindicates thestackframe(stackframezeroinafunctioncalledClearBlock()atvirtualaddress0x00012ac4). Thefollowinglineprecededby43isthelinenumberoftheoffendingsourcelinefromafilecalled led.c.Fromthere,GDBdisplaysitscommandpromptandwaitsforinput. To provide some context, we enter the gdb list command, using its abbreviated form l. GDB recognizes command abbreviations where there is no ambiguity. Here the program error begins to presentitself.Theoffendingline,accordingtoGDB'sanalysisofthecoredumpis: 43*ptr=0; Nextweissuethegdbprintcommandontheptrvariable,againabbreviatedasp.Asyoucansee from Listing 13-1, the value of the pointer ptr is 0. So we conclude that the reason for the segmentationfaultisthedereferenceofanullpointer,acommonprogrammingerror.Fromhere,we canelecttousethebacktracecommandtoseethecallchainleadingtothiserror,whichmightleadus backtotheactualsourceoftheerror.Listing13-2displaystheseresults. Listing13-2.BacktraceCommand (gdb)bt #00x00012ac4inClearBlock(RealBigBlockPtr=0x0,l=100000000)atled.c:43 #10x00012b08inInitBlock(ptr=0x0,n=100000000)atled.c:48 #20x00012b50inErrorInHandler(wp=0x325c8,urlPrefix=0x2f648"/Error", webDir=0x2f660"",arg=0,url=0x34f30"/Error",path=0x34d68"/Error", query=0x321d8"")atled.c:61 #30x000126ccinwebsUrlHandlerRequest(wp=0x325c8)athandler.c:273 #40x0001f518inwebsGetInput(wp=0x325c8,ptext=0xbefffc40, pnbytes=0xbefffc38)atwebs.c:664 #50x0001ede0inwebsReadEvent(wp=0x325c8)atwebs.c:362 #60x0001ed34inwebsSocketEvent(sid=1,mask=2,iwp=206280)atwebs.c:319 #70x00019740insocketDoEvent(sp=0x34fc8)atsockGen.c:903 #80x00019598insocketProcess(sid=1)atsockGen.c:845 #90x00012be8inmain(argc=1,argv=0xbefffe14)atmain.c:99 (gdb) Thebacktracedisplaysthecallchainallthewaybacktomain(),thestartoftheuser'sprogram.A stack frame number precedes each line of the backtrace. You can switch to any given stack frame usingthegdbframecommand.Listing13-3isanexampleofthis.Hereweswitchtostackframe2 anddisplaythesourcecodeinthatframe.Asinthepreviousexamples,thelinesprecededwith(gdb) arethecommandsweissuetoGDB,andtheotherlinesaretheGDBoutput. Listing13-3.MovingAroundStackFramesinGDB (gdb)frame2 #20x00012b50inErrorInHandler(wp=0x325c8,urlPrefix=0x2f648"/Error", webDir=0x2f660"",arg=0,url=0x34f30"/Error",path=0x34d68"/Error", query=0x321d8"")atled.c:61 61returnInitBlock(p,siz); (gdb)l 56 57siz=10000*sizeof(BigBlock); 58 59p=malloc(siz); 60/*if(p)*/ 61returnInitBlock(p,siz); 62/*elsereturn(0);*/ 63} 64 65 (gdb) Asyoucansee,withalittlehelpfromthesourcecodeavailableusingthelistcommand,itwould notbedifficulttotracethecodebacktothesourceoftheerrantnullpointer.Infact,theastutereader willnoticethesourceofthesegmentationfaultwehaveproducedforthisexample.FromListing133,weseethatthecheckofthereturnvalueinthecalltomalloc()hasbeencommentedout.Inthis example,themalloc()callfailed,leadingtotheoperationonanullpointertwoframeslaterinthe call chain. Although this example is both contrived and trivial, many crashes of this type are remarkablyeasytotrackdownusingasimilarmethodwithGDBandcoredumps.Youcanalsosee thenullpointerbylookingattheparametervaluesinthefunctioncall.Thisoftenleadsyoudirectlyto theframewherethenullpointeroriginated. 13.1.3.DebugSessioninGDB We conclude this introduction to GDB by showing a typical debug session. In the previous demonstrationofaprogramcrash,wecouldhaveelectedtostepthroughthecodetonarrowdown thecauseofthefailure.Ofcourse,ifyougetacoredump,youshouldalwaysstartthere.However,in othersituations,youmightwanttosetbreakpointsandstepthroughrunningcode.Listing13-4details how we start GDB in preparation for a debug session. Note that the program must have been compiledwiththedebugflagenabledinthegcccommandlineforGDBtobeusefulinthiscontext. Refer back to Figure 12-1 in Chapter 12, "Embedded Development Environment"; this is a crossdebug session with GDB running on your development host, debugging a program running on your target. We cover complete details of remote application debugging in Chapter 15, "Debugging EmbeddedLinuxApplications." Listing13-4.InitiatingaGDBDebugSession $xscale_be-gdb-silentwebs (gdb)targetremote192.168.1.21:2001 0x40000790in??() (gdb)bmain Breakpoint1at0x12b74:filemain.c,line78. (gdb)c Continuing. Breakpoint1,main(argc=1,argv=0xbefffe04)atmain.c:78 78bopen(NULL,(60*1024),B_USE_MALLOC); (gdb)bErrorInHandler Breakpoint2at0x12b30:fileled.c,line57. (gdb)c Continuing. Breakpoint2,ErrorInHandler(wp=0x311a0,urlPrefix=0x2f648"/Error", webDir=0x2f660"",arg=0,url=0x31e88"/Error",path=0x31918"/Error", query=0x318e8"")atled.c:57 57siz=10000*sizeof(BigBlock); (gdb)next 59p=malloc(siz); (gdb)next 61returnInitBlock(p,siz); (gdb)pp $1=(unsignedchar*)0x0 (gdb)psiz $2=100000000 (gdb) Followingthroughthissimpledebugsession,firstweconnecttoourtargetboardusingthegdb targetcommand.WecoverremotedebugginginmoredetailinChapter15.Whenweareconnectedto our target hardware, we set a breakpoint at main() using the gdb break (abbreviated b) command. Thenweissuethegdbcontinue(abbreviatedc)commandtoresumeexecutionoftheprogram.Ifwe hadanyprogramarguments,wecouldhaveissuedthemonthecommandlinewhenweinvokedGDB. We hit the breakpoint set at main(), and set another one at ErrorInHandler(), followed by the continuecommand,againabbreviated.Whenthisnewbreakpointishit,webegintostepthroughthe codeusingthenextcommand.Thereweencounterthecalltomalloc().Followingthemalloc()call, weexaminethereturnvalueanddiscoverthefailureasindicatedbythenullreturnvalue.Finally,we print the value of the parameter in the malloc() call and see that a very large memory region (100 millionbytes)isbeingrequested,whichfails. Although trivial, the GDB examples in this section should enable the newcomer to become immediatelyproductivewithGDB.FewofushavereallymasteredGDBitisverycomplexandhas many capabilities. Later in Section 13.2, "Data Display Debugger," we introduce a graphical front endtoGDBthatcaneasethetransitionforthoseunfamiliarwithGDB. OnefinalnoteaboutGDB:NodoubtyouhavenoticedthemanybannerlinesGDBdisplayson theconsolewhenitisfirstinvoked,asinListing13-1.Intheseexamples,asstatedearlier,weuseda cross-gdbfromtheMontaVistaembeddedLinuxdistribution.Thebannerlinescontainavitalpiece ofinformationthattheembeddeddevelopermustbeawareof:GDB'shostandtargetspecifications. FromListing13-1,wesawthefollowingoutputwhenGDBwasinvoked: This GDB was configured as "--host=i686-pc-linux-gnu -- target=armv5teb-montavistalinuxeabi" Inthisinstance,wewereinvokingaversionofGDBthatwascompiledtoexecutefromaLinux PCspecifically, an i686 running the GNU/Linux operating system. Equally critical, this instance of GDBwascompiledtodebugARMbinarycodegeneratedfromthearmv5tebbigendiantoolchain. Oneofthemostcommonmistakesmadebynewcomerstoembeddeddevelopmentistousethe wrong GDB while trying to debug target executables. If something isn't working right, you should immediately checkyourGDB configurationtomakesurethatitmakes senseforyourenvironment. YoucannotuseyournativeGDBtodebugtargetcode! 13.2.DataDisplayDebugger The Data Display Debugger (DDD) is a graphical front end to GDB and other command line debuggers. DDD has many advanced features beyond simply viewing source code and stepping throughadebugsession.Figure13-1isascreenshotoftheDDD'smainscreen. Figure13-1.DataDisplayDebugger DDDisinvokedasfollows: $ddd--debuggerxscale_be-gdbwebs Withoutthe--debuggerflag,DDDwouldattempttoinvokethenativeGDBonyourdevelopment host,whichisnotwhatyouwantifyouareplanningtodebuganapplicationonyourtargetsystem. ThesecondargumentontheDDDcommandlineistheprogramyouwillbedebugging.Seetheman pageforDDDforadditionaldetails. UsingthecommandtoolasshowninFigure13-1,youcanstepthroughyourprogram.Youcanset breakpointseithergraphicallyorviatheGDBconsolewindowatthebottomoftheDDDscreen.For targetdebugging,youmustfirstconnectyourdebuggertothetargetsystemaswedidinListing13-4, usingthetargetcommand.ThiscommandisissuedintheGDBwindowofthedddmainscreen. When you are connected to the target, you can execute similar commands to the sequence described in the previous example to isolate the program failure. Figure 13-2 shows the DDD displayduringthelaterphaseofthisdebuggingsession. Figure13-2.DebugsessioninDDD NoticethatinFigure13-2wehaveinitiatedthedisplayofsomeimportantprogramvariablesthat can help us narrow the cause of the segmentation fault. We can watch these variables as we step throughtheprogramusingthecommandtoolshowninthefigure. DDDisapowerfulgraphicalfrontendforGDB.Itisrelativelyeasytouseandwidelysupported formanydevelopmenthosts.ConsultSection13.7.1attheendofthischapterforalinktotheGNU DDDdocumentation. 13.3.cbrowser/cscope WementioncbrowserherebecausesupportforthishandytoolhasfounditswayintotheLinux kernelsourcetree.[82]cbrowserisasimplesource-codebrowsingtoolthatmakesiteasytobounce aroundalargesourcetreefollowingsymbols. TheLinuxkernelmakefilesupportsbuildingthedatabasethatcbrowseruses.Hereisanexample invocationfromarecentLinuxkernelsnapshot: $makeARCH=ppcCROSS_COMPILE=ppc_82xx-cscope Thisproducesthecscopesymboldatabasethatcbrowseruses.cscopeistheengine;cbrowseris thegraphicaluserinterface.Youcanusecscopeonitsownifyouwant.Itiscommandlinedriven andverypowerful,butnotquiteasquickoreasyfornavigatingalargesourcetreeinthispoint-andclickera.Ifviisstillyourfavoriteeditor,cscopemightbejustforyou! Toinvokecbrowser,enterthedirectorythatcontainsyourcscopedatabase,andsimplytypethe cbrowsercommandwithoutarguments.Figure13-3showsanexamplesession.Youcanreadmore aboutbothoftheseusefultoolsinthereferenceslistedinSection13.7.1attheendofthischapter. Figure13-3.cbrowserinaction 13.4.TracingandProfilingTools Many useful tools can provide you with various views of the system. Some tools offer a highlevel perspective, such as what processes are running on your system and which processes are consuming the most CPU bandwidth. Other tools can provide detailed analysis, such as where memory is being allocated or, even more useful, where it is being leaked. The next few sections introduce the most important tools and utilities in this category. We have space for only a cursory introductiontothesetools;referencesareprovidedwhereappropriateifyouwantmoredetails. 13.4.1.strace This useful system trace utility is found in virtually all Linux distributions. strace captures and displays useful information for every kernel system call executed by a Linux application program. straceisespeciallyhandybecauseitcanberunonprogramsforwhichnosourcecodeisavailable.It isnotnecessarytocompiletheprogramwithdebugsymbolsasitiswithGDB.Furthermore,strace canbeaveryinsightfuleducationaltool.Asthemanpagestates,"Students,hackersandtheoverlycuriouswillfindthatagreatdealcanbelearnedaboutasystemanditssystemcallsbytracingeven ordinaryprograms." WhilepreparingtheexamplesoftwarefortheGDBsectionearlierinthischapter,Idecidedto use a software project unfamiliar to me, an early version of the GoAhead web server. The first attempt at compiling and linking the project led to an interesting example for strace. Starting the applicationfromthecommandlinesilentlyreturnedcontrolbacktotheconsole.Noerrormessages wereproduced,andalookintothesystemlogsalsoproducednoclues!Itsimplywouldnotrun. stracequicklyidentifiedtheproblem.Theoutputfrominvokingstraceonthissoftwarepackageis producedinListing13-5.Manylinesfromthisoutputhavebeendeletedduetospaceconsiderations. Theuneditedoutputisoveronehundredlineslong. Listing13-5.[83]straceOutput:GoAheadWebDemo 01root@coyote:/home/websdemo$strace./websdemo 02execve("./websdemo",["./websdemo"],[/*14vars*/])=0 03uname({sys="Linux",node="coyote",...})=0 04brk(0)=0x10031050 05open("/etc/ld.so.preload",O_RDONLY)=-1ENOENT(Nosuchfileordirectory) 06open("/etc/ld.so.cache",O_RDONLY)=-1ENOENT(Nosuchfileordirectory) 07open("/lib/libc.so.6",O_RDONLY)=3 08read(3,"\177ELF\1\2\1\0\0\0\0\0\0\0\0\0\0\3\0\24\0\0\0\1\0\1\322"...,1024)=1024 09fstat64(0x3,0x7fffefc8)=0 10mmap(0xfe9f000,1379388,PROT_READ|PROT_EXEC,MAP_PRIVATE,3,0)=0xfe9f000 11mprotect(0xffd8000,97340,PROT_NONE)=0 12 mmap(0xffdf000, 61440, PROT_READ|PROT_WRITE|PROT_EXEC,MAP_PRIVATE|MAP_FIXED, 3, 0x130000) = 0xffdf000 13 mmap(0xffee000, 7228, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS,-1,0)=0xffee000 14close(3)=0 15brk(0)=0x10031050 16brk(0x10032050)=0x10032050 17brk(0x10033000)=0x10033000 18brk(0x10041000)=0x10041000 19rt_sigaction(SIGPIPE,{SIG_IGN},{SIG_DFL},8)=0 20stat("./umconfig.txt",0x7ffff9b8)=-1ENOENT(Nosuchfileordirectory) 21uname({sys="Linux",node="coyote",...})=0 22gettimeofday({3301,178955},NULL)=0 23getpid()=156 24open("/etc/resolv.conf",O_RDONLY)=3 25fstat64(0x3,0x7fffd7f8)=0 26 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,0)=0x30017000 27read(3,"#\n#resolv.confThisfileisth"...,4096)=83 28read(3,"",4096)=0 29close(3)=0 ...<<<Lines30-81removedforbrevity 82socket(PF_INET,SOCK_DGRAM,IPPROTO_IP)=3 83connect(3,{sa_family=AF_INET,sin_port=htons(53),sin_addr=inet_addr("0.0.0.0")},28)= 0 84send(3,"\267s\1\0\0\1\0\0\0\0\0\0\6coyotea\0\0\1\0\1",24,0)=24 85gettimeofday({3301,549664},NULL)=0 86poll([{fd=3,events=POLLIN,revents=POLLERR}],1,5000)=1 87ioctl(3,0x4004667f,0x7fffe6a8)=0 88recvfrom(3,0x7ffff1f0,1024,0,0x7fffe668,0x7fffe6ac)=-1ECONNREFUSED(Connection refused) 89close(3)=0 90socket(PF_INET,SOCK_DGRAM,IPPROTO_IP)=3 91connect(3,{sa_family=AF_INET,sin_port=htons(53),sin_addr=inet_addr("0.0.0.0")},28)= 0 92send(3,"\267s\1\0\0\1\0\0\0\0\0\0\6coyote\0\0\1\0\1",24,0)=24 93gettimeofday({3301,552839},NULL)=0 94poll([{fd=3,events=POLLIN,revents=POLLERR}],1,5000)=1 95ioctl(3,0x4004667f,0x7fffe6a8)=0 96recvfrom(3,0x7ffff1f0,1024,0,0x7fffe668,0x7fffe6ac)=-1ECONNREFUSED(Connection refused) 97close(3)=0 98exit(-1)=? 99root@coyote:/home/websdemo# Line numbers have been added to the output produced by strace to make this listing more readable.Invocationofthecommandisfoundonlinenumber01.Initssimplestform,simplyaddthe strace command directly in front of the program you want to examine. This is how the output in Listing13-5wasproduced. Eachlineofthistracerepresentsthewebsdemoprocessmakingasystemcallintothekernel.We don'tneedtoanalyzeandunderstandeachlineofthetrace,althoughitisquiteinstructivetodoso. We are looking for any anomalies that might help pinpoint why the program won't run. In the first severallines,Linuxissettinguptheenvironmentinwhichtheprogramwillexecute.Weseeseveral open()systemcallsto/etc/ld.so.*,whicharetheLinuxdynamiclinker-loader(ld.so)doingitsjob.In fact,line06wasmycluethatthisexampleembeddedboardhadnotbeenproperlyconfigured.There should be a linker cache produced by running ldconfig. (The linker cache substantially speeds up searchingforsharedlibraryreferences.)Thiswassubsequentlyresolvedbyrunningldconfigonthe target. Down through line 19 is more basic housekeeping, mostly by the loader and libc initializing. Noticeinline20thattheprogramislookingforaconfigurationfilebutdidnotfindone.Thatcould beanimportantissuewhenwegetthesoftwarerunning.Startingwithline24,theprogrambeginsto setupandconfiguretheappropriatenetworkingresourcesthatitneeds.Lines24through29openand read a Linux system file containing instructions for the DNS service to resolve hostnames. Local network configuration activity continues through line 81. Most of this activity consists of network setupandconfigurationnecessarytobuildthenetworkinginfrastructurefortheprogramitself.This portionofthelistinghasbeenremovedforbrevityandclarity. Noticeespeciallythenetworkactivitystartingwithline82.Herewehavetheprogramtryingto establish a TCP/IP connection to an IP address of all zeros. Line 82 is reproduced here for convenience: socket(PF_INET,SOCK_DGRAM,IPPROTO_IP)=3 AcouplepointsaboutListing13-5areworthnoting.Wemightnotknowallthedetailsofevery systemcall,butwecangetageneralideaofwhatishappening.Thesocket()systemcallissimilarto afilesystemopen()call.Thereturnvalue,indicatedbythe=sign,inthiscase,representsaLinux filedescriptor.Knowingthis,wecanassociatetheactivityfromline82throughtheclose()system callinline89withfiledescriptor3. Weareinterestedinthisgroupofrelatedsystemcallsbecauseweseeanerrormessageinline 88: "Connection refused." At this point, we still don't know why the program won't run, but this appearsabnormal.Let'sinvestigate.Line82,thesystemcalltosocket(),establishesanendpointfor IP communication. Line 83 is quite curious because it tries to establish a connection to a remote endpoint (socket) containing an IP address of all zeros. We don't have to be network experts to suspect that this might be causing trouble.[84] Line 83 provides another important clue: The port parameterissetto53.AquickGooglesearchforTCP/IPportnumbersrevealsthatport53is the DomainNameService,orDNS. Line84providesyetanotherclue.Ourboardhasahostnameofcoyote.Thiscanbeseenaspart ofthecommandpromptinline01ofListing13-5.ItappearsthatthisactivityisaDNSlookupforour board's hostname, which is failing. As an experiment, we add an entry in our target system's /etc/hosts[85] file to associate our locally defined hostname with the board's IP locally assigned IP address,asfollows: Coyote192.168.1.21#TheIPaddressweassigned Voilà: Our program begins to function normally. Although we might not know exactly why this wouldleadtoaprogramfailure(TCP/IPnetworkingexpertsmight),ourstraceoutputledustothe factthataDNSlookupforourboardnamewasfailing.Whenwecorrectedthat,theprogramstarted uphappilyandbeganservingwebpages.Torecap,thiswasaprogramforwhichwehadnosource codetoreference,andithadnosymbolscompiledintoitsbinaryimage.Usingstrace,wewereable todeterminethecauseoftheprogramfailure,andimplementasolution. 13.4.2.straceVariations Thestraceutilityhasmanycommandlineoptions.Oneofthemoreusefulincludesthecapability toselectasubsetofsystemcallsfortracing.Forexample,ifyouwanttoseeonlythenetwork-related activityofagivenprocess,issuethecommandasfollows: $strace-etrace=networkprocess_name This produces a trace of all the network-related system calls, such as socket(), connect(), recvfrom(), and send(). This is a powerful way to view the network activity of a given program. Severalothersubsetsareavailable.Forexample,youcanviewonlythefile-relatedactivitiesofa program,withopen(),close(),read(),write(),andsoon.Additionalsubsetsincludeprocess-related systemcalls,signal-relatedsystemcalls,andIPC-relatedsystemcalls. Itisworthnotingthatstraceiscapableofdealingwithtracingprogramsthatspawnadditional processes. Invoking strace with the -f option instructs strace to follow child processes that are createdusingthefork()systemcall.Numerouspossibilitiesexistwiththestracecommand.Thebest waytobecomeproficientwiththispowerfulutilityistouseit.Makeitapointwiththisandallthe toolswepresenttoseekoutandreadthelatestopen-sourcedocumentation.Inthiscase,manstrace onmostLinuxhostswillproduceenoughmaterialtokeepyouexperimentingforanafternoon! Oneveryusefulwaytoemploystraceisusingthe-coption.Thisoptionproducesahigh-level profilingofyourapplication.Usingthe-coption,straceaccumulatesstatisticsoneachsystemcall, howmanytimesitwasencountered,howmanytimeserrorswerereturned,andthetimespentineach system call. Listing 13-6 is an example of running strace -c on the webs demo from the previous example. Listing13-6.ProfilingUsingstrace root@coyote$strace-c./webs %timesecondsusecs/callcallserrorssyscall -----------------------------------------------------29.800.034262189181send 18.460.02122610112110open 14.110.016221130125read 11.870.013651506278stat64 5.880.00676219335select 5.280.0060727680fcntl64 3.470.0039946561time 2.790.00320532051execve 1.710.00197090223recv 1.620.0018688522close 1.610.00185616911shutdown 1.380.00158614411accept 0.410.000470945mmap2 0.260.0003011003mprotect 0.240.000281943brk 0.170.00019419411access 0.130.0001501501lseek 0.120.000141473uname 0.110.0001321321listen 0.110.0001281281socket 0.090.000105532fstat64 0.080.000097971munmap 0.060.000064641getcwd 0.050.000063631bind 0.050.000054541setsockopt 0.040.000048481rt_sigaction 0.040.000046461gettimeofday 0.030.000038381getpid --------------------------------------------------------100.000.11498562422total Thisisaveryusefulwaytogetahigh-levelviewofwhereyourapplicationisconsumingtime andwhereerrorsareoccurring.Someerrorsmightbeanormalpartofyourapplication'soperation, butothersmightbeconsumingtimethatyouhadn'tintended.FromListing13-6,wecanseethatthe syscallwiththelongestdurationwastheexecve(),whichisthecallthattheshellusedtospawnthe application. As you can see, it was called only once. Another interesting observation is that the send()systemcallwasthemostfrequentlyusedsyscall.Thismakessensetheapplicationisasmall webserver. Bearinmindthat,liketheothertoolswehavebeendiscussinghere,stracemustbecompiledfor yourtargetarchitecture.straceisexecutedonyourtargetboard,notyourdevelopmenthost.Youmust use a version that is compatible with your architecture. If you purchase a commercial embedded Linuxdistribution,youshouldmakesurethatthisutilityisincludedforyourchosenarchitecture. 13.4.3.ltrace The ltrace and strace utilities are closely related. The ltrace utility does for library calls what stracedoesforsystemcalls.Itisinvokedinasimilarfashion:Precedetheprogramtobetracedby thetracerutility,asfollows: $ltrace./example Listing13-7reproducestheoutputofltraceonasmallexampleprogramthatexecutesahandful ofstandardClibrarycalls. Listing13-7.ExampleltraceOutput $ltrace./example __libc_start_main(0x8048594,1,0xbffff944,0x80486b4,0x80486fc<unfinished...> malloc(256)=0x804a008 getenv("HOME")="/home/chris" strncpy(0x804a008,"/home",5)=0x804a008 fopen("foo.txt","w")=0x804a110 printf("$HOME=%s\n","/home/chris"$HOME=/home/chris )=20 fprintf(0x804a110,"$HOME=%s\n","/home/chris")=20 fclose(0x804a110)=0 remove("foo.txt")=0 free(0x804a008)=<void> +++exited(status0)+++ $ For each library call, the name of the call is displayed, along with varying portions of the parameterstothecall.Similartostrace,thereturnvalueofthelibrarycallisthendisplayed.Aswith strace,thistoolcanbeusedonprogramsforwhichsourcecodeisnotavailable. Aswithstrace,avarietyofswitchesaffectthebehaviorofltrace.Youcandisplaythevalueof the program counter at each library call, which can be helpful in understanding your application's program flow. As with strace, you can use -c to accumulate and report count, error, and time statistics, making a useful simple profiling tool. Listing 13-8 displays the results of our simple exampleprogramusingthe-coption. Listing13-8.ProfilingUsingltrace $ltrace-c./example $HOME=/home/chris %timesecondsusecs/callcallsfunction ----------------------------------------------------24.160.0002312311printf 16.530.0001581581fclose 16.000.0001531531fopen 13.700.0001311311malloc 10.670.0001021021remove 9.310.000089891fprintf 3.350.000032321getenv 3.140.000030301free 3.140.000030301strncpy ----------------------------------------------------100.000.0009569total Theltracetoolisavailableonlyforprogramsthathavebeencompiledtousedynamicallylinked shared library objects. This is the usual default, so unless you explicitly specify -static when compiling,youcanuseltraceontheresultingbinary.Againsimilartostrace,youmustuseanltrace binarythathasbeencompiledforyourtargetarchitecture.Theseutilitiesarerunonthetarget,notthe hostdevelopmentsystem. 13.4.4.ps With the possible exception of strace and ltrace, no tools are more often neglected by the embeddedsystemsdeveloperthantopandps.Giventhemyriadoptionsavailableforeachutility,we could easily devote an entire chapter to these useful system-profiling tools. They are almost universallyavailableinembeddedLinuxdistributions. Both of these utilities make use of the /proc file system, as described in Chapter 9, "File Systems."Muchoftheinformationtheyconveycanbelearnedfromthe/procfilesystemifyouknow whattolookforandhowtoparsetheresultinginformation.Thesetoolspresentthatinformationina convenienthuman-readableform. Thepsutilitylistsalltherunningprocessesonamachine.However,itisveryflexibleandcanbe tailoredtoprovidemuchusefuldataonthestateofarunningmachineandtheprocessesrunningonit. For example, ps can display the scheduling policy of each process. This is particularly useful for systemsthatemployreal-timeprocesses. Withoutanyoptions,psdisplaysallprocesseswiththesameuserIDastheuserwhoinvokedthe command,andonlythoseprocessesassociatedwiththeterminalonwhichthecommandwasissued. Thisisusefulwhenmanyjobshavebeenspawnedbythatuserandterminal. Passing options to ps can be confusing because ps supports a wide variety of standards (as in POSIX versus UNIX) and three distinct options styles: BSD, UNIX, and GNU. In general, BSD options are single or multiple letters, with no dash. UNIX options are the familiar dash-letter combinations, and GNU uses long argument formats preceded by double dashes. Refer to the man pagefordetailsofyourpsimplementation. Everyonewhousespslikelyhasafavoriteinvocation.Oneparticularlyusefulgeneral-purpose invocationispsaux.Thisdisplayseveryprocessonthesystem.Listing13-9isanexamplefroma runningembeddedtargetboard. Listing13-9.ProcessListing $psaux USERPID%CPU%MEMVSZRSSTTYSTATSTARTTIMECOMMAND root10.00.81416508?S00:000:00init[3] root20.00.000?S<00:000:00[ksoftirqd/0] root30.00.000?S<00:000:00[desched/0] root40.00.000?S<00:000:00[events/0] root50.00.000?S<00:000:00[khelper] root100.00.000?S<00:000:00[kthread] root210.00.000?S<00:000:00[kblockd/0] root620.00.000?S00:000:00[pdflush] root630.00.000?S00:000:00[pdflush] root650.00.000?S<00:000:00[aio/0] root360.00.000?S00:000:00[kapmd] root640.00.000?S00:000:00[kswapd0] root6170.00.000?S00:000:00[mtdblockd] root6380.00.000?S00:000:00[rpciod] bin8340.00.71568444?Ss00:000:00/sbin/portmap root8610.00.000?S00:000:00[lockd] root8680.00.91488596?Ss00:000:00/sbin/syslogd-r root8760.00.71416456?Ss00:000:00/sbin/klogd-x root8840.01.11660700?Ss00:000:00/usr/sbin/rpc.statd root8960.00.91668584?Ss00:000:00/usr/sbin/inetd root9090.02.224121372?Ss+00:000:00-bash telnetd9530.31.11736732?S05:580:00in.telnetd root9540.22.123841348pts/0Ss05:580:00-bash root9600.01.22312772pts/0R+05:590:00psaux Thisisbutoneofthemanywaystoviewoutputdatausingps.Thecolumnsareexplainedinthe followingtext. •TheUSERandprocessID(PID)fieldsshouldbeself-explanatory. •The%CPUfieldexpressesthepercentofCPUutilizationsincethebeginningoftheprocess's lifetime;thus,CPUusagewillvirtuallyneveraddupto100percent. • The %MEM field indicates the ratio of the process's resident memory footprint to the total availablephysicalmemory. •TheVSZfieldisthevirtualmemorysizeoftheprocessinkilobytes. •RSSisresidentsetsizeandindicatesthenonswappedphysicalmemorythataprocesshasused, alsoinkilobytes. •TTYisthecontrollingterminaloftheprocess. Most of the processes in this example are not associated with a controlling terminal. The ps command that generated Listing 13-9 was issued from a Telnet session, which is indicated by the pts/0terminaldevice. TheSTATfielddescribesthestateoftheprocessatthetimethissnapshotwasproduced.Here,S means that the process is sleeping, waiting on an event of some type, often I/O. R means that the processisinarunnablestate(thatis,theschedulerisfreetogiveitcontroloftheCPUifnothingofa higherpriorityiswaiting).Theleftbracketnexttothestateletterisanindicationthatthisprocesshas ahigherpriority. Thefinalcolumnisthecommandname.Thoselistedinbracketsarekernelthreads.Manymore symbolsandoptionsareavailable;refertothemanpageforpsforcompletedetails. 13.4.5.top Whereaspsisaone-timesnapshotofthecurrentsystem,toptakesperiodicsnapshotsofthestate of the system and its processes. Similar to ps, top has numerous command line and configuration options. It is interactive and can be reconfigured while operating to customize the display to your particularneeds. Enteredwithoutoptions,topdisplaysallrunningprocessesinafashionverysimilartothepsaux commandpresentedinListing13-9,updatedevery3seconds.Ofcourse,thisandmanyotheraspects oftopare user configurable.Thefirstfewlinesofthetopscreendisplaysysteminformation,also updatedevery3 seconds.Thisincludesthesystemuptime,thenumberofusers,informationonthe numberofprocessesandtheirstate,andmuchmore. Listing 13-10 shows top in its default configuration, resulting from executing top from the commandlinewithoutparameters. Listing13-10.top top-06:23:14up6:23,2users,loadaverage:0.00,0.00,0.00 Tasks:24total,1running,23sleeping,0stopped,0zombie Cpu(s):0.0%us,0.3%sy,0.0%ni,99.7%id,0.0%wa,0.0%hi,0.0%si Mem:62060ktotal,17292kused,44768kfree,0kbuffers Swap:0ktotal,0kused,0kfree,11840kcached PIDUSERPRNIVIRTRESSHRS%CPU%MEMTIME+COMMAND 978root1601924952780R0.31.50:01.22top 1root1601416508452S0.00.80:00.47init 2root5-10000S0.00.00:00.00ksoftirqd/0 3root5-10000S0.00.00:00.00desched/0 4root-2-5000S0.00.00:00.00events/0 5root10-5000S0.00.00:00.09khelper 10root18-5000S0.00.00:00.00kthread 21root20-5000S0.00.00:00.00kblockd/0 62root200000S0.00.00:00.00pdflush 63root150000S0.00.00:00.00pdflush 65root19-5000S0.00.00:00.00aio/0 36root250000S0.00.00:00.00kapmd 64root250000S0.00.00:00.00kswapd0 617root250000S0.00.00:00.00mtdblockd 638root150000S0.00.00:00.34rpciod 834bin1501568444364S0.00.70:00.00portmap 861root200000S0.00.00:00.00lockd 868root1601488596504S0.01.00:00.11syslogd 876root1901416456396S0.00.70:00.00klogd 884root1801660700612S0.01.10:00.02rpc.statd 896root1601668584504S0.00.90:00.00inetd 909root150241213721092S0.02.20:00.34bash 953telnetd1601736736616S0.01.20:00.27in.telnetd 954root150238413481096S0.02.20:00.16bash The default columns from Listing 13-10 are the PID, the user, the process priority, the process nice value, the virtual memory used by the process, the resident memory footprint, the amount of sharedmemoryusedbythetask,andotherfieldsthatareidenticaltothosedescribedintheprevious psexample. Spacepermitsonlyacursoryintroductiontotheseusefulutilities.Youareencouragedtospend anafternoonwiththemanpagesfortopandpstoexploretherichnessoftheircapabilities. 13.4.6.mtrace Themtracepackageisasimpleutilitythatanalyzesandreportsoncallstomalloc(),realloc(), and free() in your application. It is easy to use and can potentially help spot trouble in your application.Aswithotheruserlandtoolswehavebeendescribinginthischapter,youmusthavethe mtrace package configured and compiled for your architecture. mtrace is a malloc replacement librarythatisinstalledonyourtarget.Yourapplicationenablesitwithaspecialfunctioncall.Your embeddedLinuxdistributionshouldcontainthemtracepackage. Todemonstratethisutility,wecreatedasimpleprogramthatcreatesdynamicdataonasimple linkedlist.Eachlistitemwasdynamicallygenerated,aswaseachdataitemweplacedonthelist. Listing13-11reproducesthesimpleliststructure. Listing13-11.SimpleLinearLinkedList structblist_s{ structblist_s*next; char*data_item; intitem_size; intindex; }; Eachlistitemwasdynamicallycreatedusingmalloc()asfollowsandsubsequentlyplacedatthe endofthelinkedlist: structblist_s*p=malloc(sizeof(structblist_s)); Each variable-sized data item in the list was also dynamically generated and added to the list itembeforebeingplacedattheendofthelist.Thisway,everylistitemwascreatedusingtwocalls to malloc(), one for the list item itself, represented by struct blist_s just shown, and one for the variable data item. We then generated 10,000 records on the list containing variable string data, resultingin20,000callstomalloc(). Tousemtrace,threeconditionsmustbesatisfied: •Aheaderfile,mcheck.h,mustbeincludedinthesourcefile. •Theapplicationmustcallmtrace()toinstallthehandlers. • The environment variable MALLOC_TRACE must specify the name of a writeable file to whichthetracedataiswritten. Whentheseconditionsaresatisfied,eachcalltooneofthetracedfunctionsgeneratesalineinthe rawtracefiledefinedbyMALLOC_TRACE.Thetracedatalookslikethis: @./mt_ex:[0x80486ec]+0x804a5f80x10 The @ sign signals that the trace line contains an address or function name. In the previous example, the program was executing at the address in square brackets, 0x80486ec. Using binary utilities or a debugger, we could easily associate this address with a function. The plus sign (+) indicatesthatthisisacalltoallocatememory.Acalltofree()wouldbeindicatedbyaminussign. Thenextfieldindicatesthevirtualaddressofthememorylocationbeingallocatedorfreed.Thelast fieldisthesize,whichisincludedineverycalltoallocatememory. Thisdataformatisnotveryuserfriendly.Forthisreason,themtracepackageincludesautility[86] that analyzes the raw trace data and reports on any inconsistencies. In the simplest case, the Perl script simply prints a single line with the message "No memory leaks". Listing 13-12 contains the outputwhenmemoryleaksaredetected. Listing13-12.mtraceErrorReport $mtrace./mt_exmtrace.log Memorynotfreed: ----------------AddressSizeCaller 0x0804aa700x0aat/home/chris/temp/mt_ex.c:64 0x0804abc00x10at/home/chris/temp/mt_ex.c:26 0x0804ac600x10at/home/chris/temp/mt_ex.c:26 0x0804acc80x0aat/home/chris/temp/mt_ex.c:64 Asyoucansee,thissimpletoolcanhelpyouspottroublebeforeithappens,aswellasfindit whenitdoes.NoticethatthePerlscripthasdisplayedthefilenameandlinenumberofeachcallto malloc() that does not have a corresponding call to free() for the given memory location. This requiresdebugginginformationintheexecutablefilegeneratedbypassingthe-gflagtothecompiler. If no debugging information is found, the script simply reports the address of the function calling malloc(). 13.4.7.dmalloc dmalloc picks up where mTRace leaves off. The mtrace package is a simple, relatively nonintrusive package most useful for simple detection of malloc /free unbalance conditions. The dmalloc package enables the detection of a much wider range of dynamic memory-management errors.ComparedtomTRace,dmallocishighlyintrusive.Dependingontheconfiguration,dmalloc canslowyourapplicationtoacrawl.Itisdefinitelynottherighttoolifyoususpectmemoryerrors duetoraceconditionsorothertimingissues.dmalloc(andmtrace,toalesserextent)willdefinitely changethetimingofyourapplication. dmalloc is a very powerful dynamic memory-analysis tool. It is highly configurable and, therefore, somewhatcomplex. Ittakessometimetolearnandmasterthistool.However, fromQA testingtobugsquashing,itcouldbecomeoneofyourfavoritedevelopmenttools. dmallocisadebugmalloclibraryreplacement.Theseconditionsmustbesatisfiedtousedmalloc : •Applicationcodemustincludethedmalloc.hheaderfile. •Theapplicationmustbelinkedagainstthedmalloclibrary. •Thedmalloclibraryandutilitymustbeinstalledonyourembeddedtarget. • Certain environment variables that the dmalloc library references must be defined before runningyourapplicationonthetarget. Althoughitisnotstrictlynecessary,youshouldincludedmalloc.hinyourapplicationprogram. Thisallowsdmalloctoincludefileandlinenumberinformationintheoutput. Link your application against the dmalloc library of your choice. The dmalloc package can be configured to generate several different libraries, depending on your selections during package configuration. In the examples to follow, we have chosen to use the libdmalloc.so shared library object.Placethelibrary(orasymlinktoit)inapathwhereyourcompilercanfindit.Thecommand tocompileyourapplicationmightlooksomethinglikethis: $ppc_82xx-gcc-g-Wall-omtest_ex-L../dmalloc-5.4.2/\ -ldmallocmtest_ex.c Thiscommandlineassumesthatyou'veplacedthedmalloclibrary(libdmalloc.so)inalocation searched by the -L switch on the command linenamely, the ../dmalloc-5.4.2 directly just above the currentdirectory. To install the dmalloc library on your target, place it in your favorite location (perhaps /usr/local/lib). You might need to configure your system to find this library. On our example PowerPCsystem,weaddedthepath/usr/local/libtothe/etc/ld.so.conffileandinvokedtheldconfig utilitytoupdatethelibrarysearchcache. The last step in preparation is to set an environment variable that the dmalloc library uses to determinethelevelofdebuggingthatwillbeenabled.Theenvironmentvariablecontainsadebugbit mask that concatenates a number of features into a single convenient variable. Yours might look somethinglikethis: DMALLOC_OPTIONS=debug=0x4f4ed03,inter=100,log=dmalloc.log Here, debug is the debug-level bit mask, and inter sets an interval count at which the dmalloc libraryperformsextensivechecksonitselfandtheheap.Thedmalloclibrarywritesitslogoutputto thefileindicatedbythelogvariable. The dmalloc package comes with a utility to generate the DMALLOC_OPTIONS environment variable based on flags passed to it. The previous example was generated with the following dmallocinvocation.Thedocumentationinthedmallocpackagedetailsthisquitethoroughly,sowe shallnotreproducethathere. $dmalloc-pcheck-fence-ldmalloc.log-i100high Whenthesestepsarecomplete,youshouldbeabletorunyourapplicationagainstthedmalloc debuglibrary. dmalloc produces a quite detailed output log. Listing 13-13 reproduces a sample dmalloc log outputforanexampleprogramthatintentionallygeneratessomememoryleaks. Listing13-13.dmallocLogOutput 2592:4002:Dmallocversion'5.4.2'from'http://dmalloc.com/' 2592:4002:flags=0x4f4e503,logfile'dmalloc.log' 2592:4002:interval=100,addr=0,seen#=0,limit=0 2592:4002:startingtime=2592 2592:4002:processpid=442 2592:4002:DumpingChunkStatistics: 2592:4002:basic-block4096bytes,alignment8bytes 2592:4002:heapaddressrange:0x30015000to0x3004f000,237568bytes 2592:4002:userblocks:18blocks,73652bytes(38%) 2592:4002:adminblocks:29blocks,118784bytes(61%) 2592:4002:totalblocks:47blocks,192512bytes 2592:4002:heapchecked41 2592:4002:alloccalls:malloc2003,calloc0,realloc0,free1999 2592:4002:alloccalls:recalloc0,memalign0,valloc0 2592:4002:alloccalls:new0,delete0 2592:4002:currentmemoryinuse:52bytes(4pnts) 2592:4002:totalmemoryallocated:27546bytes(2003pnts) 2592:4002:maxinuseatonetime:27546bytes(2003pnts) 2592:4002:maxallocedwith1call:376bytes 2592:4002:maxunusedmemoryspace:37542bytes(57%) 2592:4002:top10allocations: 2592:4002:total-sizecountin-use-sizecountsource 2592:4002:160001000322mtest_ex.c:36 2592:4002:108901000202mtest_ex.c:74 2592:4002:256100mtest_ex.c:154 2592:4002:271462001524Totalof3 2592:4002:DumpingNot-FreedPointersChangedSinceStart: 2592:4002:notfreed:'0x300204e8|s1'(10bytes)from'mtest_ex.c:74' 2592:4002:notfreed:'0x30020588|s1'(16bytes)from'mtest_ex.c:36' 2592:4002:notfreed:'0x30020688|s1'(16bytes)from'mtest_ex.c:36' 2592:4002:notfreed:'0x300208a8|s1'(10bytes)from'mtest_ex.c:74' 2592:4002:total-sizecountsource 2592:4002:322mtest_ex.c:36 2592:4002:202mtest_ex.c:74 2592:4002:524Totalof2 2592:4002:endingtime=2592,elapsedsincestart=0:00:00 Itisimportanttonotethatthislogisgenerateduponprogramexit.(dmallochasmanyoptionsand modes of operation; it is possible to configure dmalloc to print output lines when errors are detected.) Thefirsthalfoftheoutputlogreportshigh-levelstatisticsabouttheheapandtheoverallmemory usageoftheapplication.Totalsareproducedforeachofthemalloclibrarycalls,suchasmalloc(), free(), and realloc(). Interestingly, this default log reports on the top 10 allocations and the source locationwheretheyoccurred.Thiscanbeveryusefulforoverallsystem-levelprofiling. Towardtheendofthelog,weseeevidenceofmemoryleaksinourapplication.Youcanseethat thedmalloclibrarydetectedfourinstancesofmemorythatwasallocatedthatwasapparentlynever freed.Becauseweincludeddmalloc.handcompiledwithdebugsymbols,thesourcelocationwhere thememorywasallocatedisindicatedinthelog. Aswiththeothertoolswe'vecoveredinthischapter,spacepermitsonlyabriefintroductionof this very powerful debug tool. dmalloc can detect many other conditions and limits. For example, dmalloccandetectwhenafreedpointerhasbeenwritten.Itcantellwhetherapointerwasusedto accessdataoutsideitsboundsbutwithintheapplication'spermissibleaddressrange.Infact,dmalloc canbeconfiguredtologalmostanymemorytransactionthroughthemallocfamilyofcalls.dmallocis atoolthatissuretopaybackmanytimestheefforttakentobecomeproficientwithit. 13.4.8.KernelOops Although not strictly a tool, a kernel oops contains much useful information to help you troubleshoot the cause. A kernel oops results from a variety of kernel errors from simple memory errorsproducedbyaprocess(fullyrecoverable,inmostcases)toahardkernelpanic.RecentLinux kernelssupportdisplayofsymbolicinformationinadditiontotherawhexadecimaladdressvalues. Listing13-14reproducesakerneloopsfromaPowerPCtarget. Listing13-14.KernelOops $modprobeloop Oops:kernelaccessofbadarea,sig:11[#1] NIP:C000D058LR:C0085650SP:C7787E80REGS:c7787dd0TRAP:0300Nottainted MSR:00009032EE:1PR:0FP:0ME:1IR/DR:11 DAR:00000000,DSISR:22000000 TASK=c7d187b0[323]'modprobe'THREAD:c7786000 Lastsyscall:128 GPR00: 0000006C C7787E80 C7D187B0 00000000 C7CD25CC FFFFFFFF 00000000 80808081 GPR08: 00000001 C034AD80 C036D41C C034AD80 C0335AB0 1001E3C0 00000000 00000000 GPR16:000000000000000000000000100170D8100013E0C9040000C903DFD8C9040000 GPR24: 00000000 C9040000 C9040000 00000940 C778A000 C7CD25C0 C7CD25C0 C7CD25CC NIP[c000d058]strcpy+0x10/0x1c LR[c0085650]register_disk+0xec/0xf0 Calltrace: [c00e170c]add_disk+0x58/0x74 [c90061e0]loop_init+0x1e0/0x430[loop] [c002fc90]sys_init_module+0x1f4/0x2e0 [c00040a0]ret_from_syscall+0x0/0x44 Segmentationfault Noticethattheregisterdumpincludessymbolicinformation,whereappropriate.Yourkernelmust have KALLSYSMS enabled for this symbolic information to be available. Figure 13-4 shows the configurationoptionsundertheGeneralSetupmainmenu. Figure13-4.Symbolsupportforoops Much of the information in a kernel oops message is directly related to the processor. Having someknowledgeoftheunderlyingarchitectureisnecessarytofullyunderstandtheoopsmessage. Analyzing the oops in Listing 13-14, we see right away that the oops was generated due to a "kernelaccessofbadarea,sig:11".Wealreadyknowfrompreviousexamplesinthischapterthat signal11isasegmentationfault. Thefirstsectionisasummaryshowingthereasonfortheoops,afewimportantpointers,andthe offendingtask.InListing13-14,NIPisthenextinstructionpointer,whichisdecodedlaterintheoops message.Thispointstotheoffendingcodethatledtotheoops.LRisaPowerPCregisterandusually indicates the return address for the currently executing subroutine. SP is the stack pointer. REGS indicates the kernel address for the data structure containing the register dump data, and TRAP indicates the type of exception that this oops message relates to. Referring to the PowerPC architecturereferencemanualreferencedattheendofChapter7,"Bootloaders,"weseethataTRAP 0300isthePowerPCDataStorageInterrupt,whichistriggeredbyadatamemoryaccesserror. On the third line of the oops message, we see additional PowerPC machine registers, such as MSR (machine state register) and a decode of some of its bits. On the next line, we see the DAR (data access register), which often contains the offending memory address. The DSISR register contentscanbeusedinconjunctionwiththePowerPCarchitecturereferencetodiscovermuchdetail aboutthespecificreasonfortheexception. Anoopsmessagealsocontainsthetaskpointerandthedecodedtasknametoquicklydetermine what task or thread was running at the time of the oops. We also see a detailed processor register dump, which can be used for additional clues. Again, we need knowledge of the architecture and compiler register usage to make sense of the clues from the register values. For example, the PowerPCarchitectureusesther3registerforreturnvaluesfromCfunctions. Thelastpartoftheoopsmessageprovidesastackbacktracewithsymboldecodeifsymbolsare enabledinthekernel.Usingthisinformation,wecanconstructasequenceofeventsthatledtothe offendingcondition. Inthissimpleexample,wehavelearnedagreatdealofinformationfromthisoopsmessage.We knowthatitwasaPowerPCDataStorageException,causedbyanerrorinadatamemoryaccess(as opposedtoaninstructionfetchmemoryaccess).TheDARregistertellsusthatthedataaddressthat generatedthisexceptionwas0x0000_0000.Weknowthatthemodprobeprocessproducedtheerror. FromthebacktraceandNIP(nextinstructionpointer),weknowthatitwasinacalltostrcpy()that canbetraceddirectlybacktotheloop_init()functionintheloop.komodule,whichmodprobewas tryingtoinsertatthetimeoftheexception.Giventhisinformation,trackingdownthesourceofthis errantnullpointerdereferenceshouldbequitetrivial. 13.5.BinaryUtilities Binaryutilities,orbinutils,areacriticalcomponentofanytoolchain.Indeed,tobuildacompiler, you must first have successfully built binutils. In this section, we briefly introduce the more useful tools that the embedded developer needs to know about. As with most of the other tools in this chapter, these are cross-utilities and must be built to execute on your development host while operating on binary files targeted to your chosen architecture. Alternatively, you could compile or obtainversionsofthesetorunonyourtarget,butweassumeacross-developmentenvironmentfor theseexamples. 13.5.1.readelf ThereadelfutilityexaminesthecompositionofyourtargetELFbinaryfile.Thisisparticularly useful for building images targeted for ROM or Flash memory where explicit control of the image layout is required. It is also a great tool for learning how your toolchain builds images and for understandingtheELFfileformat. Forexample,todisplaythesymboltableinanELFimage,usethiscommand: $readelf-s<elf-image> TodiscoveranddisplayallthesectionsinyourELFimage,usethiscommand: $readelf-e<elf-image> Usethe-SflagtolistthesectionheadersinyourELFimage.Youmightbesurprisedtolearnthat evenasimpleseven-line"helloworld"programcontains38separatesections.Someofthemwillbe familiartoyou,suchasthe.textand.datasections.Listing13-15containsapartiallistingofsections fromour"helloworld"example.Forsimplicity,wehavelistedonlythosesectionsthatarelikelyto befamiliarorrelevanttotheembeddeddeveloper. Listing13-15.readelfSectionHeaders $ppc_82xx-readelf-Shello-ex Thereare38sectionheaders,startingatoffset0x32f4: SectionHeaders: [Nr]NameTypeAddrOffSizeESFlgLkInfAl ... [11].textPROGBITS100002f00002f000056800AX004 ... [13].rodataPROGBITS1000087800087800006800A004 ... [15].dataPROGBITS100108e00008e000000c00WA004 ... [22].sdataPROGBITS100109e00009e000001c00WA004 [23].sbssNOBITS100109fc0009fc00000000WA001 ... [25].bssNOBITS10010a740009fc00001c00WA004 ... The.textsectioncontainstheexecutableprogramcode.The.rodatasectioncontainsconstantdata in your program. The .data section generally contains initialized global data used by the C library prologuecodeandcancontainlargeinitializeddataitemsfromyourapplication.The.sdatasection is used for smaller initialized global data items and exists only on some architectures. Some processor architectures can make use of optimized data access when the attributes of the memory areaareknown.The.sdataand.sbsssectionsenabletheseoptimizations.The.bssand.sbsssections containuninitializeddatainyourprogram.Thesesectionsoccupynospaceintheprogramimagetheir memoryspaceisallocatedandinitializedtozeroonprogramstartupbyClibraryprologuecode. Wecandumpanyofthesesectionsanddisplaythecontents.GiventhislineinyourCprogram declaredoutsideofanyfunction,wecanexaminehowitisplacedinthe.rodatasection: char*hello_rodata="Thisisaread-onlydatastring\n"; IssuethereadelfcommandspecifyingthesectionnumberwewanttodumpfromListing13-15: $ppc_82xx-readelf-x13hello-ex Hexdumpofsection'.rodata': 0x10000878100189e0100004881000050c1000058c................ 0x1000088800020001546869732069732061207265....Thisisaread0x1000089861642d6f6e6c79206461746120737472onlydatastring 0x100008a8696e670a000000005468697320697320.....Thisis 0x100008b873746174696320646174610a00000000staticdata..... 0x100008c848656c6c6f20456d6265646465640a00HelloEmbedded.. 0x100008d825730a0025780a00%s..%x.. Weseethattheinitializedglobalvariablethatwedeclaredisrepresentedinthe.rodatasection, togetherwithalltheconstantstringsdefinedintheprogram. 13.5.2.ExaminingDebugInfoUsingreadelf OneofthemoreusefulfeaturesofreadelfistodisplaythedebuginformationcontainedinanELF file. When the -g compiler flag is issued during a compilation, the compiler generates debug informationinaseriesofsectionswithintheresultingELFfile.Wecanusereadelftodisplaythese ELFsectionheaderswithintheELFfile: $ppc-linux-readelf-Sex_sync|grepdebug [28].debug_arangesPROGBITS00000000000c380000b800008 [29].debug_pubnamesPROGBITS00000000000cf000007a00001 [30].debug_infoPROGBITS00000000000d6a00079b00001 [31].debug_abbrevPROGBITS0000000000150500020700001 [32].debug_linePROGBITS0000000000170c00035400001 [33].debug_framePROGBITS00000000001a6000008000004 [34].debug_strPROGBITS00000000001ae000014d00001 Using readelf with the --debug-dump option, we can display the contents of any one of these .debug_*sections.YouwillseehowthisinformationcanbeusefulinChapter14,"KernelDebugging Techniques,"whenwediscussthechallengeofdebuggingoptimizedkernelcode. Debug information can be very large. Displaying all the debug information in the Linux kernel ELFfilevmlinuxproducesmorethansixmillionlinesofoutput.Howeverdauntingitmightappear, havingatleastafamiliaritywithdebuginformationwillmakeyouabetterembeddedengineer. Listing13-16isapartiallistingofthecontentsofthe.debug_infosectionfromasmallexample application.Forspaceconsiderations,wehaveshownonlyafewrecords. Listing13-16.PartialDebugInfoDump $ppc-linux-readelf-debug-dump=infoex_sync 1Thesection.debug_infocontains: 2 3CompilationUnit@0: 4Length:109 5Version:2 6AbbrevOffset:0 7PointerSize:4 8<0><b>:AbbrevNumber:1(DW_TAG_compile_unit) 9DW_AT_stmt_list:0 10DW_AT_low_pc:0x10000368 11DW_AT_high_pc:0x1000038c 12DW_AT_name: ../sysdeps/powerpc/powerpc32/elf/start.S 13DW_AT_comp_dir:/var/tmp/BUILD/glibc-2.3.3/csu 14DW_AT_producer:GNUAS2.15.94 15DW_AT_language:32769(MIPSassembler) ... 394<1><5a1>:AbbrevNumber:14(DW_TAG_subprogram) 395DW_AT_sibling:<5fa> 396DW_AT_external:1 397DW_AT_name:main 398DW_AT_decl_file:1 399DW_AT_decl_line:9 400DW_AT_prototyped:1 401DW_AT_type:<248> 402DW_AT_low_pc:0x100004b8 403DW_AT_high_pc:0x10000570 404DW_AT_frame_base:1byteblock:6f(DW_OP_reg31) ... 423<2><5e9>:AbbrevNumber:16(DW_TAG_variable) 424DW_AT_name:mybuf 425DW_AT_decl_file:1 426DW_AT_decl_line:11 427DW_AT_type:<600> 428DW_AT_location:2byteblock:9120(DW_OP_fbreg:32) ... The first record identified by the Dwarf2[87] tag DW_TAG_compile_unit identifies the first compilation unit of this PowerPC executable. It is a file called start.S, which provides startup prologueforaCprogram.ThenextrecordidentifiedbyDW_TAG_subprogramidentifiesthestartof theuserprogram,thefamiliarfunctionmain().ThisDwarf2debugrecordcontainsareferencetothe file and line number where main() is found. The final record in Listing 13-16 identifies a local variable in the main() routine called mybuf. Again, the line number and file are provided by this record.Youcandeducefromthisinformationthatmain()isatline9,andmybufisatline11ofthe source file. Other debug records in the ELF file correlate the filename via the Dwarf2 DW_AT_decl_fileattribute. YoucandiscoverallthedetailsoftheDwarf2debuginformationformatviathereferencegiven inSection13.7.1attheendofthischapter. 13.5.3.objdump The objdump utility has considerable overlap with the readelf tool. However, one of the more useful features of objdump is its capability to display disassembled object code. Listing 13-17 provides an example of disassembly of the .text section of the simple "hello world" PowerPC version. We include only the main() routine, to save space. The entire dump, including C library prologueandepilogue,wouldconsumemanypages. Listing13-17.DisassemblyUsingobjdump $ppc_82xx-objdump-S-mpowerpc:common-j.texthello ... 10000488<main>: 10000488:9421ffe0stwur1,-32(r1) 1000048c:7c0802a6mflrr0 10000490:93e1001cstwr31,28(r1) 10000494:90010024stwr0,36(r1) 10000498:7c3f0b78mrr31,r1 1000049c:907f0008stwr3,8(r31) 100004a0:909f000cstwr4,12(r31) 100004a4:3d201000lisr9,4096 100004a8:38690854addir3,r9,2132 100004ac:4cc63182crclr4*cr1+eq 100004b0:48010511bl100109c0 <__bss_start+0x60> 100004b4:38000000lir0,0 100004b8:7c030378mrr3,r0 100004bc:81610000lwzr11,0(r1) 100004c0:800b0004lwzr0,4(r11) 100004c4:7c0803a6mtlrr0 100004c8:83ebfffclwzr31,-4(r11) 100004cc:7d615b78mrr1,r11 100004d0:4e800020blr ... Much of the code from the simple main() routine is stack frame creation and destruction. The actualcalltoprintf()isrepresentedbythebranchlink(bl)instructionnearthecenterofthelistingat address 0x100004b0. This is a PowerPC function call. Because this program was compiled as a dynamicallylinkedobject,wewillnothaveanaddressfortheprintf()functionuntilruntime,whenit islinkedwiththesharedlibraryprintf()routine.Hadwecompiledthisasastaticallylinkedobject, wewouldseethesymbolandcorrespondingaddressforthecalltoprintf(). 13.5.4.objcopy objcopyformatsand,optionally,convertstheformatofabinaryobjectfile.Thisutilityisquite usefulforgeneratingcodeforROMorFlashresidentimages.TheU-Bootbootloaderintroducedin Chapter7makesuseofobjcopytoproducebinaryands-record[88]outputformatsfromthefinalELF file.ThisexampleusageillustratesthecapabilitiesofobjcopyanditsusetobuildFlashimages. $ppc_82xx-objcopy--gap-fill=0xff-Obinaryu-bootu-boot.bin ThisobjcopyinvocationshowshowanimagemightbepreparedforFlashmemory.Theinputfile u-boot, in this exampleis the complete ELF U-Boot image, including symbols and relocation information. The objcopy utility takes only the relevant sections containing program code and data andplacestheimageintheoutputfile,specifiedhereasu-boot.bin. Flashmemorycontainsallonesinitserasedstate.Therefore,fillinggapsinabinaryimagewith all ones improves programming efficiency and prolongs the life of the Flash memory, which today haslimitedwritecycles.Thisisdonewiththe--gap-fillparametertoobjcopy. Thisisbutonesimpleexampleusageofobjcopy.Thisutilitycanbeusedtogenerates-records andconvertfromoneformattoanother.Seethemanpageforcompletedetails. 13.6.MiscellaneousBinaryUtilities Your toolchain contains several additional useful utilities. Learning to use these utilities is straightforward.Youwillfindmanyusesforthesehelpfultools. 13.6.1.strip The strip utility can be used to remove symbols and debug information from a binary. This is frequently used to save space on an embedded device. In the cross-development model, it is convenienttoplacestrippedbinariesonthetargetsystemandleavetheunstrippedversiononyour development host. Using this method, symbols are available for cross-debugging on your developmenthostwhilesavingspaceonthetarget.striphasmanyoptions,whicharedescribedinthe manpage. 13.6.2.addr2line WhenwehighlightedmtraceinListing13-12,yousawthattheoutputfromthemtraceanalysis scriptcontainedfileandlinenumberinformation.ThemTRacePerlscriptusedtheaddr2lineutility to read the debug information contained in the executable ELF file and display a line number correspondingtotheaddress.Usingthesamemtraceexampleexecutable,wecanfindafilenameand linenumberforavirtualaddress: $addr2line-f-emt_ex0x80487c6 put_data /home/chris/examples/mt_ex.c:64 Noticethatthefunctionput_data()isalsolistedtogetherwiththefileandlinenumber.Thissays thattheaddress0x80487c6isonline64ofthemt_ex.cfile,intheput_data()function.Thisiseven moreusefulinlargerbinariesconsistingofmultiplefilenames,suchastheLinuxkernel: $ppc_82xx-addr2line-f-evmlinuxc000d95c mpc52xx_restart arch/ppc/syslib/mpc52xx_setup.c:41 Thisparticularexamplehighlightsoneofthepointsrepeatedthroughoutthischapter:Thisisan architecture-specifictool.Youmustuseatoolconfiguredandcompiledtomatchthearchitectureof thetargetbinarythatyouareusing.Aswiththecross-compiler,addr2lineisacross-toolandpartof thebinaryutilitiespackage. 13.6.3.strings The strings utility examines ASCII string data in binary files. This is especially useful for examining memory dumps when source code or debug symbols might not be available. You might oftendiscoverthatyoucannarrowthecauseofacrashbytracingthestringsbacktotheoffending binary.Althoughstringsdoeshaveafewcommandlineoptions,itiseasytolearnanduse.Seethe manpageforfurtherdetails. 13.6.4.ldd Although not strictly a binary utility, the ldd script is another useful tool for the embedded developer.ItispartoftheClibrarypackageandexistsonvirtuallyeveryLinuxdistribution.lddlists thesharedobjectlibrarydependenciesforagivenobjectfileorfiles.WeintroducedlddinChapter 11, "BusyBox."SeeListing 11-2 foranexampleusage.Thelddscriptisparticularly useful during development of ramdisk images. One of the most common failures asked about on the various embeddedLinuxmailinglistsisakernelpanicaftermountingroot: VFS:Mountedroot(nfsfilesystem). Freeingunusedkernelmemory:96kinit Kernelpanic-notsyncing:Noinitfound.Trypassinginit=optiontokernel. Oneofthemostcommoncausesisthattherootfilesystemimage(beitramdisk,Flash,orNFS root file system) does not have the supporting libraries for the binaries that the kernel is trying to execute.Usingldd,youcandeterminewhichlibrarieseachofyourbinariesrequiresandmakesure thatyouincludetheminyourramdiskorotherrootfilesystemimage.Inthepreviousexamplekernel panic,initwasindeedonthefilesystem,buttheLinuxdynamicloader,ld.so.1,wasmissing.Using lddisquitestraightforward: $xscale_be-lddinit libc.so.6=>/opt/mvl/.../lib/libc.so.6(0xdead1000) ld-linux.so.3=>/opt/mvl/.../lib/ld-linux.so.3(0xdead2000) Thissimpleexampledemonstratesthattheinitbinaryrequirestwodynamiclibraryobjects:libc andld-linux.Bothmustbeonyourtargetandmustbeaccessibletoyourinitbinarythatis,theymust bereadableandexecutable. 13.6.5.nm Thenmutilitydisplayssymbolsfromanobjectfile.Thiscanbeusefulforavarietyoftasks.For example,whencross-compilingalargeapplication,youencounterunresolvedsymbols.Youcanuse nmtofindwhichobjectmodulecontainsthosesymbolsandthenmodifyyourbuildenvironmentto includeit. Thenmutilityprovidesattributesforeachsymbol.Forexample,youcandiscoverwhetherthis symbolislocalorglobal,orwhetheritisdefinedorreferencedonlyinaparticularobjectmodule. Listing13-18reproducesseverallinesfromtheoutputofnmrunontheU-BootELFimageu-boot. Listing13-18.DisplayingSymbolsUsingnm $ppc_85xx-nmu-boot ... fff23140bbase_address fff24c98BBootFile fff06d64TBootpRequest fff00118tboot_warm fff21010dborder fff23000A__bss_start ... Notice the link addresses of these U-Boot symbols. They were linked for a Flash device that livesinthehighestportionofthememorymaponthisparticularboard.Thislistingcontainsonlya fewexamplesymbols,fordiscussionpurposes.Themiddlecolumnisthesymboltype.Acapitalized letterindicatesaglobalsymbol,andlowercaseindicatesalocalsymbol.Bindicatesthatthesymbol islocatedinthe.bsssection.Tindicatesthatthesymbolislocatedinthe.textsection.Dindicates that the symbol is located in the .data section. A indicates that this address is absolute and is not subjecttomodificationbyanadditionallinkstage.Thisabsolutesymbolindicatesthestartofthe.bss section and is used by the code that clears the .bss on startup, as required for a C execution environment. 13.6.6.prelink The prelink utility is often used in systems in which startup time is important. A dynamically linked ELF executable must be linked at runtime when the program is first loaded. This can take significanttimeinalargeapplication.prelinkpreparesthesharedlibrariesandtheobjectfilesthat dependonthemtoprovidea-prioriknowledgeoftheunresolvedlibraryreferences.Ineffect,thiscan reducethestartuptimeofagivenapplication.Themanpagehascompletedetailsontheuseofthis handyutility. 13.7.ChapterSummary • The GNU Debugger (GDB) is a complex and powerful debugger with many capabilities. We presentedthebasicstogetyoustarted. •TheDDDgraphicalfrontendforGDBintegratessourcecodeanddatadisplaywiththepower ofGDBcommandlineinterfacecapabilities. •cbrowserisausefulaidforunderstandinglargeprojects.Itusesthecscopedatabasetorapidly findanddisplaysymbolsandotherelementsofCsourcecode. • Linuxis supported by manyprofilingandtracetools.Wepresentedseveral,includingstrace, ltrace,top,andps,andthememoryprofilersmtraceanddmalloc. •Embeddeddevelopersoftenneedtobuildcustomimagessuchasthoserequiredforbootloaders andfirmwareimages.Forthesetasks,knowledgeofbinutilsisindispensable.Wepresentedmanyof theutilitiesfoundinbinutils,includingreadelf,objdump,objcopy,andseveralothers. 13.7.1.SuggestionsforAdditionalReading GDB:TheGNUProjectDebugger: www.gnu.org/software/gdb/gdb.html GDBPocketReference ArnoldRobbins O'ReillyMedia,2005 DataDisplayDebugger: www.gnu.org/software/ddd/ cbrowserhomepage: http://cbrowser.sourceforge.net/ cscopehomepage: http://cscope.sourceforge.net/index.html dmallocDebugMallocLibrary: http://dmalloc.com/ ToolInterfaceStandard(TIS)ExecutableandLinkingFormat(ELF)Specification Version1.2 TISCommittee,May1995 Toolinterfacestandards: DWARFDebuggingInformationFormatSpecification Version2.0 TISCommittee,May1995 Chapter14.KernelDebuggingTechniques Oftenthepivotalfactorinachievingdevelopmenttimetablescomesdowntoone'sefficiencyin findingandfixingbugs.DebugginginsidetheLinuxkernelcanbequitechallenging.Nomatterhow you approach it, kernel debugging will always be complex. This chapter examines some of the complexitiesandpresentsideasandmethodstoimproveyourdebuggingskillsinsidethekerneland devicedrivers. 14.1.ChallengestoKernelDebugging Debugging a modern operating system involves many challenges. Virtual memory operating systemspresenttheirownuniquechallenges.Gonearethedayswhenwecouldreplaceaprocessor with an in-circuit emulator. Processors have become far too fast and complex. Moreover, pipeline architectureshideimportantcode-executiondetails,partlybecausememoryaccessesonthebuscan be ordered differently from code execution, and particularly because of internal caching of instructionstreams.Itisnotalwayspossibletocorrelateexternalbusactivitytointernalprocessor instructionexecution,exceptatarathercoarselevel. SomeofthechallengesyouwillencounterwhiledebuggingLinuxkernelcodeare: •Linuxkernelcodeishighlyoptimizedforspeedofexecutioninmanyareas. • Compilers use optimization techniques that complicate the correlation of C source to actual machineinstructionflow.Inlinefunctionsareagoodexampleofthis. • Single-stepping through compiler optimized code often produces unusual and unexpected results. • Virtual memory isolates user space memory from kernel memory and can make various debuggingscenariosespeciallydifficult. •Somecodecannotbesteppedthroughwithtraditionaldebuggers. •Startupcodecanbeespeciallydifficultbecauseofitsproximitytothehardwareandthelimited resourcesavailable(forexample,noconsole,limitedmemorymapping,andsoon). The Linux kernel has matured into a very high-performance operating system capable of competing with the best commercial operating systems. Many areas within the kernel do not lend themselves to easy analysis by simply reading the source code. Knowledge of the architecture and detailed design are often necessary to understand the code flow in a particular area. Several good booksareavailablethatdescribethekerneldesignindetail.RefertoSection14.6.1,"Suggestions forAdditionalReading,"forrecommendations. GCCisanoptimizingcompiler.Bydefault,theLinuxkerneliscompiledwiththe-O2compiler flag.Thisenablesmanyoptimizationalgorithmsthatcanchangethefundamentalstructureandorder ofyourcode.[89]Forexample,theLinuxkernelmakesheavyuseofinlinefunctions.Inlinefunctions are small functions declared with the inline keyword, which results in the function being included directlyintheexecutionthreadinsteadofgeneratingafunctioncallandtheassociatedoverhead.[90] Inline functions require a minimum of -O1 optimization level. Therefore, you cannot turn off optimization,whichwouldbedesirableforeasierdebugging. In many areas within the Linux kernel, single-stepping through code is difficult or impossible. The most obvious examples are code paths that modify the virtual memory settings. When your applicationmakesasystemcallthatresultsinentryintothekernel,thisresultsinachangeinaddress spaceasseenbytheprocess.Infact,anytransitionthatinvolvesaprocessorexceptionchangesthe operationalcontextandcanbedifficultorimpossibletosingle-stepthrough. 14.2.UsingKGDBforKernelDebugging Twopopularmethodsenablesymbolicsource-leveldebuggingwithintheLinuxkernel: •UsingKGDBasaremotegdbagent •UsingahardwareJTAGprobetocontroltheprocessor WecoverJTAGdebugginginSection14.4,"Hardware-AssistedDebugging." KGDB (Kernel GDB) is a set of Linux kernel patches that provide an interface to gdb via its remoteserialprotocol.KGDBimplementsagdbstubthatcommunicatestoacross-gdbrunningon your host development workstation. Until very recently, KGDB on the target required a serial connectiontothedevelopmenthost.SometargetssupportKGDBconnectionviaEthernet,although thisisrelativelynew.CompletesupportforKGDBisstillnotinthemainlinekernel.orgkernel.You needtoportKGDBtoyourchosentargetorobtainanembeddedLinuxdistributionforyourchosen architectureandplatformthatcontainsKGDBsupport.MostembeddedLinuxdistributionsavailable todaysupportKGDB. Figure 14-1 describes the KGDB debug setup. Up to three connections to the target board are used.EthernetisusedtoenableNFSrootmountandtelnetsessionsfromthehost.Ifyourboardhasa ramdiskimageinFlashthatitmountsasarootfilesystem,youcaneliminatetheEthernetconnection. Figure14-1.KGDBdebugsetup AserialportisdedicatedfortheconnectionbetweenKGBDandgdbrunningonthedevelopment hostsystem,andanoptionalsecondserialportservesasaconsole.Systemsthathaveonlyoneserial portmakeKGDBsomewhatmorecumbersometouse. As you can see in Figure 14-1, the debugger (your cross-version of gdb) runs on your development host system. KGDB is part of the kernel running on your target system. KGDB implements the hooks required to interface gdb with your target board to enable features such as settingbreakpoints,examiningmemory,andenablingsingle-stepprogramexecution. 14.2.1.KGDBKernelConfiguration KGDBisakernelfeatureandmustbeenabledinyourkernel.KGDBisselectedfromtheKernel Hackingmenu,asshowninFigure14-2.Aspartoftheconfiguration,youmustselecttheserialport forKGDBtouse.NoticealsofromFigure14-2thatweenabledtheoptiontocompilethekernelwith debuginformation.Thisaddsthe-gcompilerflagtothebuildprocesstoenablesymbolicdebugging. Figure14-2.KernelconfigurationforKGDB 14.2.2.TargetBootwithKGDBSupport AfteryourkernelisbuiltwithKGDBsupport,itmustbeenabled.Unfortunately,themethodto enable it is not yet uniform across all architectures and implementations. In general, KGDB is enabled by passing a command line switch to the kernel via the kernel command line. If KGDB supportiscompiledintothekernelbutnotenabledviaacommandlineswitch,itdoesnothing.When KGDB is enabled, the kernel stops at a KGDB-enabled breakpoint very early in the boot cycle to allow you to connect to the target using gdb. Figure 14-3 shows the logic for generating an initial breakpointwhenKGDBisenabled. Figure14-3.KGDBlogic KGDBrequiresaserialportforconnectiontothehost.[91]ThefirststepinsettingupKGDBisto enableaserialportveryearlyinthebootprocess.Inmanyarchitectures,thehardwareUARTmust bemappedintokernelmemorybeforeaccess.Aftertheaddressrangeismapped,theserialportis initialized.Debugtraphandlersareinstalledtoallowprocessorexceptionstotrapintothedebugger. Listing 14-1 displays the terminal output when booting with KGDB enabled. This example is based on the AMCC 440EP Evaluation Kit (Yosemite board), which ships with the U-Boot bootloader. Listing14-1.BootingwithKGDBEnabledUsingU-Boot =>setebootargsconsole=ttyS1,115200root=/dev/nfsrwip=dhcpgdb =>bootm200000 ##Bootingimageat00200000... ImageName:Linux-2.6.13 ImageType:PowerPCLinuxKernelImage(gzipcompressed) DataSize:1064790Bytes=1MB LoadAddress:00000000 EntryPoint:00000000 VerifyingChecksum...OK UncompressingKernelImage...OK $T0440:c000ae5c;01:c0205fa0;#d9<<<Seetext MostofthebootsequenceisfamiliarfromourcoverageofU-BootinChapter7,"Bootloaders." This kernel boot sequence has two unique features: the command-line parameter to enable KGDB andtheodd-lookingtextstringafterthekernelisuncompressed. Recall from Chapter 7 that the kernel command line is defined by the U-Boot bootargs environment variable. Notice that we have added the gdb parameter, which instructs the kernel to forceanearlybreakpointandwaitforthehostdebugger(yourcross-gdb)toconnect. AsdiagrammedinFigure14-3,thekerneldetectsthepresenceofthegdbparameterandattempts to pass control to the remote (host-based) debugger. This is evidenced by the sequence of ASCII charactersdumpedtotheserialportinListing14-1.Ifyouarecuriousaboutthisgdbremoteserial protocol,itisdocumentedinthegdbmanualcitedattheendofthischapter.Inthisexample,KGDB issendingaStopReplypacketreportingthebreakpointtraptotheremotegdbsessiononthehost. Thetwo32-bitparametersindicatethelocationoftheprogramandthestackframe. Now that the kernel is set up and waiting for the host debugger, we can begin our debugging session.Weinvokecross-gdbfromourhostdevelopmentworkstationandconnecttothetargetvia gdb 's remote protocol. In this example, we are sharing the serial port, so we must disconnect the terminalemulatorfromthetargetbeforetryingtoconnectwithgdb.Listing14-2highlightsthegdb connection process. This assumes that we have already exited our terminal emulator and freed the serialportforgdbtouse. Listing14-2.ConnectingtoKGDB $ppc_4xx-gdb--silentvmlinux (gdb)targetremote/dev/ttyS0 Remotedebuggingusing/dev/ttyS0 breakinst()atarch/ppc/kernel/ppc-stub.c:825 825} (gdb)l 820return; 821} 822 823asm(".globlbreakinst\n\ 824breakinst:.long0x7d821008"); 825} 826 827#ifdefCONFIG_KGDB_CONSOLE 828/*OutputstringinGDBO-packetformatifGDBhasconnected. Ifnothing 829output,returns0(callermustthenhandleoutput).*/ (gdb) Herewehaveperformedthreeactions: •Invokedgdb,passingitthekernelELFfilevmlinux •Connectedtothetargetusingthetargetremotecommandwithingdb •Issuedthelistcommand,usingitsabbreviatedformtodisplayourlocationinthesourcecode Attheriskofpointingouttheobvious,thevmlinuximagethatwepasstogdbmustbefromthe samekernelbuildthatproducedthetargetkernelbinary.Italsomusthavebeencompiledwiththe-g compilerflagtocontaindebuginformation. When we issued the target remote command, gdb responded by displaying the location of the programcounter(PC).Inthisexample,thekernelisstoppedatthebreakpointdefinedbytheinline assemblerstatementatline823infile.../arch/ppc/kernel/ppc-stub.c.Whenweissuethecontinue(c) command,executionresumesstartingatline825,asindicated. 14.2.3.UsefulKernelBreakpoints Wehavenowestablishedadebugconnectionwiththekernelonourtargetboard.Whenweissue the gdb continue (c) command, the kernel proceeds to boot, and if there are no problems, the boot process completes. There is one minor limitation of using KGDB on many architectures and processors. An engineering trade-off was made between the need to support very early kernel debugging(forexample,beforeafull-blowninterrupt-drivenserialportdriverisinstalled)andthe desiretokeepthecomplexityoftheKGDBdebugengineitselfverysimpleand,therefore,robustand portable.KGDBusesasimplepolledserialdriverthathaszerooverheadwhenthekernelisrunning. Asadrawbacktothisimplementation,thetraditionalCtl-CorBreaksequenceontheserialportwill have no effect. Therefore, it will be impossible to stop execution of the running kernel unless a breakpointorotherfaultisencountered. Forthisreason,ithasbecomecommonpracticetodefinesomesystem-widebreakpoints,which provide the capability to halt the current thread of execution. Two of the most common are highlightedinListing14-3. Listing14-3.CommonKernelBreakpoints (gdb)bpanic Breakpoint1at0xc0016b18:filekernel/panic.c,line74. (gdb)bsys_sync Breakpoint2at0xc005a8c8:filefs/buffer.c,line296. (gdb) Using the gdb breakpoint command, again using its abbreviated version, we enter two breakpoints.Oneisatpanic()andtheotherisatthesyncsystemcallentrysys_sync().Theformer allowsthedebuggertobeinvokedifalatereventgeneratesapanic.Thisenablesexaminationofthe systemstateatthetimeofthepanic.Thesecondisausefulwaytohaltthekernelandtrapintothe debugger from user space by entering the sync command from a terminal running on your target hardware. We are now ready to proceed with our debugging session. We have a KGDB-enabled kernel runningonourtarget,pausedataKGDB-definedearlybreakpoint.Weestablishedaconnectionto the target with our host-based cross debuggerin this case, invoked as ppc_4xx-gdb and we have enteredapairofusefulsystembreakpoints.Nowwecandirectourdebuggingactivitiestothetaskat hand. One caveat: By definition, we cannot use KGDB for stepping through code before the breakpoint() function in .../arch/ppc/setup.c used to establish the connection between a KGDBenabled kernel and cross-gdb on our host. Figure 14-3 is a rough guide to the code that executes beforeKGDBgainscontrol.Debuggingthisearlycoderequirestheuseofahardware-assisteddebug probe.WecoverthistopicshortlyinSection14.4,"Hardware-AssistedDebugging." 14.3.DebuggingtheLinuxKernel One of the more common reasons you might find yourself stepping through kernel code is to modifyorcustomizetheplatform-specificcodeforyourcustomboard.Let'sseehowthismightbe doneusingtheAMCCYosemiteboard.Weplaceabreakpointattheplatform-specificarchitecture setup function and then continue until that breakpoint is encountered. Listing 14-4 shows the sequence. Listing14-4.DebuggingArchitecture-SetupCode (gdb)byosemite_setup_arch Breakpoint3at0xc021a488: filearch/ppc/platforms/4xx/yosemite.c,line308. (gdb)c Continuing. Can'tsendsignalstothisremotesystem.SIGILLnotsent. Breakpoint3,yosemite_setup_arch()atarch/ppc/platforms/4xx/yosemite.c:308 308 yosemite_set_emacdata(); (gdb)l 303} 304 305staticvoid__init 306yosemite_setup_arch(void) 307{ 308yosemite_set_emacdata(); 309 310ibm440gx_get_clocks(&clocks,YOSEMITE_SYSCLK,6*1843200); 311ocp_sys_info.opb_bus_freq=clocks.opb; 312 (gdb) Whenthebreakpointatyosemite_setup_arch()isencountered,controlpassestogdbatline308 ofyosemite.c.Thelist(l)commanddisplaysthesourcelistingcenteredonthebreakpointatline308. Thewarningmessagedisplayedbygdbafterthecontinue(c)commandcanbesafelyignored.Itis part of gdb 's way of testing the capabilities of the remote system. It first sends a remote continue_with_signal command to the target. The KGDB implementation for this target board does not support this command; therefore, it is NAK 'd by the target. gdb responds by displaying this informationalmessageandissuingthestandardremotecontinuecommandinstead. 14.3.1.gdbRemoteSerialProtocol gdbincludesadebugswitchthatenablesustoobservetheremoteprotocolbeingusedbetween gdbonyourdevelopmenthostandthetarget.Thiscanbeveryusefulforunderstandingtheunderlying protocol, as well as troubleshooting targets that exhibit unusual or errant behavior. To enable this debugmode,issuethefollowingcommand: (gdb)setdebugremote1 Withremotedebuggingenabled,itisinstructivetoobservethecontinuecommandinactionand the steps taken by gdb. Listing 14-5 illustrates the use of the continue command with remote debuggingenabled. Listing14-5.continueRemoteProtocolExample (gdb)c Continuing. Sendingpacket:$mc0000000,4#80...Ack Packetreceived:c022d200 Sendingpacket:$Mc0000000,4:7d821008#68...Ack Packetreceived:OK Sendingpacket:$mc0016de8,4#f8...Ack Packetreceived:38600001 Sendingpacket:$Mc0016de8,4:7d821008#e0...Ack Packetreceived:OK Sendingpacket:$mc005bd5c,4#23...Ack Packetreceived:38600001 Sendingpacket:$Mc005bd5c,4:7d821008#0b...Ack Packetreceived:OK Sendingpacket:$mc021a488,4#c8...Ack Packetreceived:4bfffbad Sendingpacket:$Mc021a488,4:7d821008#b0...Ack Packetreceived:OK Sendingpacket:$c#63...Ack <<<programrunning,gdbwaitingforevent Althoughitmightlookdauntingatfirst,whatishappeninghereiseasilyunderstood.Insummary, gdb is restoring all its breakpoints on the target. Recall from Listing 14-3 that we entered two breakpoints,oneatpanic()andoneatsys_sync().LaterinListing14-4,weaddedathirdbreakpoint at yosemite_setup_arch(). Thus, there are three active user-specified breakpoints. These can be displayedbyissuingthegdbinfobreakpointscommand.Asusual,weusetheabbreviatedversion. (gdb)ib NumTypeDispEnbAddressWhat 1breakpointkeepy0xc0016de8inpanicatkernel/panic.c:74 2breakpointkeepy0xc005bd5cinsys_syncatfs/buffer.c:296 3 breakpoint keep y 0xc021a488 in yosemite_setup_arch at arch/ppc/platforms/4xx /yosemite.c:308 breakpointalreadyhit1time (gdb) Nowcomparethepreviousbreakpointaddresseswiththeaddressesinthegdbremote$mpacket inListing14-5.The$mpacketisa"readtargetmemory"command,andthe$Mpacketisa"write targetmemory"command.Onceforeachbreakpoint,theaddressofthebreakpointisreadfromtarget memory,storedawaylocallyonthehostbygdb(soitcanberestoredlater),andreplacedwiththe PowerPCTRapinstructiontwger2,r2(0x7d821008),whichresultsincontrolpassingbacktothe debugger.Figure14-4illustratesthisaction. Figure14-4.gdbinsertingtargetmemorybreakpoints Youmighthavenoticedthatgdbisupdatingfourbreakpoints,whereasweenteredonlythree.The firstoneattargetmemorylocation0xc000_0000isputtherebygdbautomaticallyuponstartup.This location is the base address of the linked kernel image from the ELF fileessentially, _start. It is equivalenttoabreakpointatmain()foruserspacedebuggingandisdonebygdbautomatically.The otherthreebreakpointsaretheonesweenteredearlier. Thesamethinghappensinreversewhenaneventoccursthatreturnscontroltogdb.Listing14-6 detailstheactionwhenourbreakpointatyosemite_setup_arch()isencountered. Listing14-6.RemoteProtocol:BreakpointHit Packetreceived:T0440:c021a488;01:c020ff90; Sendingpacket:$mc0000000,4#80...Ack<<<Readmemory@c0000000 Packetreceived:7d821008 Sendingpacket:$Mc0000000,4:c022d200#87...Ack<<<Writememory Packetreceived:OK Sendingpacket:$mc0016de8,4#f8...Ack Packetreceived:7d821008 Sendingpacket:$Mc0016de8,4:38600001#a4...Ack Packetreceived:OK Sendingpacket:$mc005bd5c,4#23...Ack Packetreceived:7d821008 Sendingpacket:$Mc005bd5c,4:38600001#cf...Ack Packetreceived:OK Sendingpacket:$mc021a488,4#c8...Ack Packetreceived:7d821008 Sendingpacket:$Mc021a488,4:4bfffbad#d1...Ack Packetreceived:OK Sendingpacket:$mc021a484,c#f3...Ack Packetreceived:900100244bfffbad3fa0c022 Breakpoint3,yosemite_setup_arch()atarch/ppc/platforms/4xx/yosemite.c:308 308yosemite_set_emacdata(); (gdb) The $T packet is a gdb Stop Reply packet. It is sent by the target to gdb when a breakpoint is encountered.Inourexample,the$Tpacketreturnedthevalueoftheprogramcounterandregisterr1. [92] The rest of the activity is the reverse of that in Listing 14-5. The PowerPC trap breakpoint instructions are removed, and gdb restores the original instructions to their respective memory locations. 14.3.2.DebuggingOptimizedKernelCode Atthestartofthischapter,wesaidthatoneofthechallengesidentifiedindebuggingkernelcode results from compiler optimization. We noted that the Linux kernel is compiled by default with optimization level -O2. In the examples up to this point, we used -O1 optimization to simplify the debuggingtask.Hereweillustrateoneofthemanywaysoptimizationcancomplicatedebugging. The related Internet mail lists are strewn with questions related to what appear to be broken tools. Sometimes the poster reports that his debugger is single-stepping backward or that his line numbers do not line up with his source code. Here we present an example to illustrate the complexities that optimizing compilers bring to source-level debugging. In this example, the line numbersthatgdbreportswhenabreakpointishitdonotmatchupwiththelinenumbersinoursource fileduetofunctioninlining. Forthisdemonstration,weusethesamedebugcodesnippetasshowninListing14-4.However, forthisexample,wehavecompiledthekernelwiththecompileroptimizationflag-O2.Thisisthe defaultfortheLinuxkernel.Listing14-7showstheresultsofthisdebuggingsession. Listing14-7.OptimizedArchitecture-SetupCode $ppc_44x-gdb--silentvmlinux (gdb)targetremote/dev/ttyS0 Remotedebuggingusing/dev/ttyS0 breakinst()atarch/ppc/kernel/ppc-stub.c:825 825} (gdb)bpanic Breakpoint1at0xc0016b18:filekernel/panic.c,line74. (gdb)bsys_sync Breakpoint2at0xc005a8c8:filefs/buffer.c,line296. (gdb)byosemite_setup_arch Breakpoint3at0xc020f438:filearch/ppc/platforms/4xx/yosemite.c,line116. (gdb)c Continuing. Breakpoint3,yosemite_setup_arch() atarch/ppc/platforms/4xx/yosemite.c:116 116def=ocp_get_one_device(OCP_VENDOR_IBM,OCP_FUNC_EMAC,0); (gdb)l 111structocp_def*def; 112structocp_func_emac_data*emacdata; 113 114/*Setmac_addrandphymodeforeachEMAC*/ 115 116def=ocp_get_one_device(OCP_VENDOR_IBM,OCP_FUNC_EMAC,0); 117emacdata=def->additions; 118memcpy(emacdata->mac_addr,__res.bi_enetaddr,6); 119emacdata->phy_mode=PHY_MODE_RMII; 120 (gdb)pyosemite_setup_arch $1={void(void)}0xc020f41c<yosemite_setup_arch> Referring back to Listing 14-4, notice that the function yosemite_setup_arch() actually falls on line306ofthefileyosemite.c.ComparethatwithListing14-7.Wehitthebreakpoint,butgdbreports thebreakpointatfileyosemite.cline116.Itappearsatfirstglancetobeamismatchoflinenumbers betweenthedebuggerandthecorrespondingsourcecode.Isthisagdbbug?Firstlet'sconfirmwhat the compiler produced for debug information. Using the readelf[93] tool described in Chapter 13, "Development Tools," we can examine the debug information for this function produced by the compiler. $ppc_44x-readelf--debug-dump=infovmlinux|grep-u6yosemite_setup_arch|tail-n7 DW_AT_name:(indirectstring,offset:0x9c04):yosemite_setup_arch DW_AT_decl_file:1 DW_AT_decl_line:307 DW_AT_prototyped:1 DW_AT_low_pc:0xc020f41c DW_AT_high_pc:0xc020f794 DW_AT_frame_base:1byteblock:51(DW_OP_reg1) Wedon'thavetobeexpertsatreadingDWARF2debugrecords[94]torecognizethatthefunction inquestionisreportedatline307inoursourcefile.Wecanconfirmthisusingtheaddr2lineutility, alsointroducedinChapter13.UsingtheaddressderivedfromgdbinListing14-7: $ppc_44x-addr2line-evmlinux0xc020f41c arch/ppc/platforms/4xx/yosemite.c:307 At this point, gdb is reporting our breakpoint at line 116 of the yosemite.c file. To understand whatishappening,weneedtolookattheassembleroutputofthefunctionasreportedbygdb.Listing 14-8 is the output from gdb after issuing the disassemble command on the yosemite_setup_arch() function. Listing14-8.DisassembleFunctionyosemite_setup_arch (gdb)disassembleyosemite_setup_arch 0xc020f41c<yosemite_setup_arch+0>:mflrr0 0xc020f420<yosemite_setup_arch+4>:stwur1,-48(r1) 0xc020f424<yosemite_setup_arch+8>:lir4,512 0xc020f428<yosemite_setup_arch+12>:lir5,0 0xc020f42c<yosemite_setup_arch+16>:lir3,4116 0xc020f430<yosemite_setup_arch+20>:stmwr25,20(r1) 0xc020f434<yosemite_setup_arch+24>:stwr0,52(r1) 0xc020f438<yosemite_setup_arch+28>:bl0xc000d344 <ocp_get_one_device> 0xc020f43c<yosemite_setup_arch+32>:lwzr31,32(r3) 0xc020f440<yosemite_setup_arch+36>:lisr4,-16350 0xc020f444<yosemite_setup_arch+40>:lir28,2 0xc020f448<yosemite_setup_arch+44>:addir4,r4,21460 0xc020f44c<yosemite_setup_arch+48>:lir5,6 0xc020f450<yosemite_setup_arch+52>:lisr29,-16350 0xc020f454<yosemite_setup_arch+56>:addir3,r31,48 0xc020f458<yosemite_setup_arch+60>:lisr25,-16350 0xc020f45c<yosemite_setup_arch+64>:bl0xc000c708 <memcpy> 0xc020f460<yosemite_setup_arch+68>:stwr28,44(r31) 0xc020f464<yosemite_setup_arch+72>:lir4,512 0xc020f468<yosemite_setup_arch+76>:lir5,1 0xc020f46c<yosemite_setup_arch+80>:lir3,4116 0xc020f470<yosemite_setup_arch+84>:addir26,r25,15104 0xc020f474<yosemite_setup_arch+88>:bl0xc000d344 <ocp_get_one_device> 0xc020f478<yosemite_setup_arch+92>:lisr4,-16350 0xc020f47c<yosemite_setup_arch+96>:lwzr31,32(r3) 0xc020f480<yosemite_setup_arch+100>:addir4,r4,21534 0xc020f484<yosemite_setup_arch+104>:lir5,6 0xc020f488<yosemite_setup_arch+108>:addir3,r31,48 0xc020f48c<yosemite_setup_arch+112>:bl0xc000c708 <memcpy> 0xc020f490<yosemite_setup_arch+116>:lisr4,1017 0xc020f494<yosemite_setup_arch+120>:lisr5,168 0xc020f498<yosemite_setup_arch+124>:stwr28,44(r31) 0xc020f49c<yosemite_setup_arch+128>:orir4,r4,16554 0xc020f4a0<yosemite_setup_arch+132>:orir5,r5,49152 0xc020f4a4<yosemite_setup_arch+136>:addir3,r29,-15380 0xc020f4a8<yosemite_setup_arch+140>:addir29,r29,-15380 0xc020f4ac<yosemite_setup_arch+144>:bl0xc020e338 <ibm440gx_get_clocks> 0xc020f4b0<yosemite_setup_arch+148>:lir0,0 0xc020f4b4<yosemite_setup_arch+152>:lisr11,-16352 0xc020f4b8<yosemite_setup_arch+156>:orir0,r0,50000 0xc020f4bc<yosemite_setup_arch+160>:lwzr10,12(r29) 0xc020f4c0<yosemite_setup_arch+164>:lisr9,-16352 0xc020f4c4<yosemite_setup_arch+168>:stwr0,8068(r11) 0xc020f4c8<yosemite_setup_arch+172>:lwzr0,84(r26) 0xc020f4cc<yosemite_setup_arch+176>:stwr10,8136(r9) 0xc020f4d0<yosemite_setup_arch+180>:mtctrr0 0xc020f4d4<yosemite_setup_arch+184>:bctrl 0xc020f4d8<yosemite_setup_arch+188>:lir5,64 0xc020f4dc<yosemite_setup_arch+192>:mrr31,r3 0xc020f4e0<yosemite_setup_arch+196>:lisr4,-4288 0xc020f4e4<yosemite_setup_arch+200>:lir3,0 0xc020f4e8<yosemite_setup_arch+204>:bl0xc000c0f8 <ioremap64> Endofassemblerdump. (gdb) Onceagain,weneednotbePowerPCassemblylanguageexpertstounderstandwhatishappening here. Notice the labels associated with the PowerPC bl instruction. This is a function call in PowerPC mnemonics. The symbolic function labels are the important data points. After a cursory analysis,weseeseveralfunctioncallsnearthestartofthisassemblerlisting: Address Function 0xc020f438 ocp_get_one_device() 0xc020f45c memcpy() 0xc020f474 ocp_get_one_device() 0xc020f48c memcpy() 0xc020f4ac ibm440gx_get_clocks() Listing14-9reproducesportionsofthesourcefileyosemite.c.Correlatingthefunctionswefound in the gdb disassemble output, we see those labels occurring in the function yosemite_set_emacdata(), around the line numbers reported by gdb when the breakpoint at yosemite_setup_arch() was encountered. The key to understanding the anomaly is to notice the subroutine call at the very start of yosemite_setup_arch(). The compiler has inlined the call to yosemite_set_emacdata() instead of generating a function call, as would be expected by simple inspectionofthesourcecode.Thisinliningproducedthemismatchinthelinenumberswhengdbhit thebreakpoint.Eventhoughtheyosemite_set_emacdata()functionwasnotdeclaredusingtheinline keyword,GCCinlinedthefunctionasaperformanceoptimization. Listing14-9.PortionsofSourceFileyosemite.c 109staticvoid__inityosemite_set_emacdata(void) 110{ 111structocp_def*def; 112structocp_func_emac_data*emacdata; 113 114/*Setmac_addrandphymodeforeachEMAC*/ 115 116def=ocp_get_one_device(OCP_VENDOR_IBM,OCP_FUNC_EMAC,0); 117emacdata=def->additions; 118memcpy(emacdata->mac_addr,__res.bi_enetaddr,6); 119emacdata->phy_mode=PHY_MODE_RMII; 120 121def=ocp_get_one_device(OCP_VENDOR_IBM,OCP_FUNC_EMAC,1); 122emacdata=def->additions; 123memcpy(emacdata->mac_addr,__res.bi_enet1addr,6); 124emacdata->phy_mode=PHY_MODE_RMII; 125} 126 ... 304 305staticvoid__init 306yosemite_setup_arch(void) 307{ 308yosemite_set_emacdata(); 309 310ibm440gx_get_clocks(&clocks,YOSEMITE_SYSCLK,6*1843200); 311ocp_sys_info.opb_bus_freq=clocks.opb; 312 313/*inittosome~sanevalueuntilcalibrate_delay()runs*/ 314loops_per_jiffy=50000000/HZ; 315 316/*SetupPCIhostbridge*/ 317yosemite_setup_hose(); 318 319#ifdefCONFIG_BLK_DEV_INITRD 320if(initrd_start) 321ROOT_DEV=Root_RAM0; 322else 323#endif 324#ifdefCONFIG_ROOT_NFS 325ROOT_DEV=Root_NFS; 326#else 327ROOT_DEV=Root_HDA1; 328#endif 329 330yosemite_early_serial_map(); 331 332/*Identifythesystem*/ 333printk("AMCCPowerPC"BOARDNAME"Platform\n"); 334} 335 Tosummarizethepreviousdiscussion: •Weenteredabreakpointingdbatyosemite_setup_arch(). •Whenthebreakpointwashit,wefoundourselvesatline116ofthesourcefile,whichwasfar removedfromthefunctionwherewedefinedthebreakpoint. • We produced a disassembly listing of the code at yosemite_setup_arch() and discovered the labelstowhichthissequenceofcodewasbranching. •Comparingthelabelsbacktooursourcecode,wediscoveredthatthecompilerhadplacedthe yosemite_set_emacdata()subroutineinlinewiththefunctionwhereweenteredabreakpoint,causing potentialconfusion. This explains the line numbers reported by gdb when the original breakpoint in yosemite_setup_arch()washit. Compilersemploymanydifferentkindsofoptimizationalgorithms.Thisexamplepresentedbut one:functioninlining.Eachcanconfuseadebugger(thehumanandthemachine)inadifferentway. Thechallengeistounderstandwhatishappeningatthemachinelevelandtranslatethatintowhatwe asdevelopershadintended.Youcanseenowthebenefitsofusingtheminimumpossibleoptimization levelfordebugging. 14.3.3.gdbUser-DefinedCommands Youmightalreadyrealizethatgdblooksforaninitializationfileonstartup,called.gdbinit.When firstinvoked,gdbloadsthisinitializationfile(usuallyfoundintheuser'shomedirectory)andactson thecommandswithinit.Oneofmyfavoritecombinationsistoconnecttothetargetsystemandset initialbreakpoints.Inthiscase,thecontentsof.gdbinitwouldlooklikeListing14-10. Listing14-10.SimplegdbInitializationFile $cat~/.gdbinit sethistorysaveon sethistoryfilename~/.gdb_history setoutput-radix16 defineconnect #targetremotebdi:2001 targetremote/dev/ttyS0 bpanic bsys_sync end Thissimple.gdbinitfileenablesthestoringofcommandhistoryinauser-specifiedfileandsets thedefault output radixfor printing of values.Thenitdefinesagdbuser-definedcommandcalled connect.(User-definedcommandsarealsooftencalledmacros.)Whenissuedatthegdb command prompt,gdbconnectstothetargetsystemviathedesiredmethodandsetsthesystembreakpointsat panic() and sys_sync(). One method is commented out; we discuss this method shortly in Section 14.4. Thereisnoendtothecreativeuseofgdbuser-definedcommands.Whendebugginginthekernel, it is often useful to examine global data structures such as task lists and memory maps. Here we presentseveralusefulgdbuser-definedcommandscapableofdisplayingspecifickerneldatathatyou mightneedtoaccessduringyourkerneldebugging. 14.3.4.UsefulKernelgdbMacros Duringkerneldebugging,itisoftenusefultoviewtheprocessesthatarerunningonthesystem,as well as some common attributes of those processes. The kernel maintains a linked list of tasks described by struct task_struct. The address of the first task in the list is contained in the kernel globalvariableinit_task,whichrepresentstheinitialtaskspawnedbythekernelduringstartup.Each taskcontainsastructlist_head,whichlinksthetasksinacircularlinkedlist.Thesetwoubiquitous kernelstructuresaredescribedinthefollowingheaderfiles: structtask_struct.../include/linux/sched.h structlist_head.../include/linux/list.h Usinggdbmacros,wecantraversethetasklistanddisplayusefulinformationaboutthetasks.It iseasytomodifythemacrostoextractthedatayoumightbeinterestedin.Itisalsoaveryusefultool forlearningthedetailsofkernelinternals. Thefirstmacroweexamine(inListing14-11)isasimpleonethatsearchesthekernel'slinked listoftask_structstructuresuntilitfindsthegiventask.Ifitisfound,itdisplaysthenameofthetask. Listing14-11.gdbfind_taskMacro 1#HelperfunctiontofindataskgivenaPIDorthe 2#addressofatask_struct. 3#Theresultissetinto$t 4definefind_task 5#Addressesgreaterthan_end:kerneldata... 6#...userpassedinanaddress 7if((unsigned)$arg0>(unsigned)&_end) 8set$t=(structtask_struct*)$arg0 9else 10#UserenteredanumericPID 11#Walkthetasklisttofindit 12set$t=&init_task 13if(init_task.pid!=(unsigned)$arg0) 14find_next_task$t 15while(&init_task!=$t&&$t->pid!=(unsigned)$arg0) 16find_next_task$t 17end 18if($t==&init_task) 19printf"Couldn'tfindtask;usinginit_task\n" 20end 21end 22end 23printf"Task\"%s\":\n",$t->comm 24end Placethistextintoyour.gdbinitfileandrestartgdb,orsource[95]itusinggdb'ssourcecommand. (Weexplainthefind_next_taskmacrolaterinListing14-15.)Invokeitasfollows: (gdb)find_task910 Task"syslogd": or (gdb)find_task0xCFFDE470 Task"bash": Line4definesthemacroname.Line7decideswhethertheinputargumentisaPID(numericentry startingatzeroandlimitedtoafewmillion)oratask_structaddressthatmustbegreaterthantheend of the Linux kernel image itself, defined by the symbol _end.[96] If it's an address, the only action required is to cast it to the proper type to enable dereferencing the associated task_struct. This is done at line 8. As the comment in line 3 states, this macro returns a gdb convenience variable typecastedtoapointertoastructtask_struct. IftheinputargumentisanumericPID,thelististraversedtofindthematchingtask_struct.Lines 12 and 13 initialize the loop variables (gdb does not have a for statement in its macro command language),andlines15through17definethesearchloop.Thefind_next_taskmacroisusedtoextract thepointertothenexttask_structinthelinkedlist.Finally,ifthesearchfails,asanereturnvalueis set(theaddressofinit_task)sothatitcanbesafelyusedinothermacros. Buildingonthefind_taskmacroinListing14-11,wecaneasilycreateasimplepscommandthat displaysusefulinformationabouteachprocessrunningonthesystem. Listing14-12definesagdbmacrothatdisplaysinterestinginformationfromarunningprocess, extractedfromthestructtask_structforthegivenprocess.Itisinvokedlikeanyothergdbcommand, bytypingitsnamefollowedbyanyrequiredinputparameters.Noticethatthisuser-definedcommand requiresasingleargument,eitheraPIDortheaddressofatask_struct. Listing14-12.gdbMacro:PrintProcessInformation 1defineps 2#Printcolumnheaders 3task_struct_header 4set$t=&init_task 5task_struct_show$t 6find_next_task$t 7#Walkthelist 8while&init_task!=$t 9#Displayusefulinfoabouteachtask 10task_struct_show$t 11find_next_task$t 12end 13end 14 15documentps 16Printpointsofinterestforalltasks 17end Thispsmacroissimilartothefind_taskmacro,exceptthatitrequiresnoinputargumentsandit adds a macro (task_struct_show) to display the useful information from each task_struct. Line 3 printsabannerlinewithcolumnheadings.Lines4through6setuptheloopanddisplaythefirsttask. Lines8through11loopthrougheachtask,callingthetask_struct_showmacroforeach. Noticealsotheinclusionofthegdbdocumentcommand.Thisallowsthegdbusertogethelpby issuingthehelppscommandfromthegdbcommandpromptasfollows: (gdb)helpps Printpointsofinterestforalltasks Listing14-13displaystheoutputofthismacroonatargetboardrunningonlyminimalservices. Listing14-13.gdbpsMacroOutput (gdb)ps AddressPIDStateUser_NIPKernel-SPdevicecomm 0xC01D37500Running0xC0205E90(none)swapper 0xC04ACB101Sleeping0x0FF6E85C0xC04FFCE0(none)init 0xC04AC7702Sleeping0xC0501E90(none)ksoftirqd/0 0xC04AC3D03Sleeping0xC0531E30(none)events/0 0xC04AC0304Sleeping0xC0533E30(none)khelper 0xC04CDB305Sleeping0xC0535E30(none)kthread 0xC04CD79023Sleeping0xC06FBE30(none)kblockd/0 0xC04CD3F045Sleeping0xC06FDE50(none)pdflush 0xC04CD05046Sleeping0xC06FFE50(none)pdflush 0xC054B7B048Sleeping0xC0703E30(none)aio/0 0xC054BB5047Sleeping0xC0701E20(none)kswapd0 0xC054B410629Sleeping0xC0781E60(none)kseriod 0xC054B070663Sleeping0xCFC59E30(none)rpciod/0 0xCFFDE0D0675Sleeping0x0FF6E85C0xCF86DCE0(none)udevd 0xCF95B110879Sleeping0x0FF0BE580xCF517D80(none)portmap 0xCFC24090910Sleeping0x0FF6E85C0xCF61BCE0(none)syslogd 0xCF804490918Sleeping0x0FF66C7C0xCF65DD70(none)klogd 0xCFE350B0948Sleeping0x0FF0E85C0xCF67DCE0(none)rpc.statd 0xCFFDE810960Sleeping0x0FF6E85C0xCF5C7CE0(none)inetd 0xCFC24B70964Sleeping0x0FEEBEAC0xCF64FD80(none)mvltd 0xCFE35B90973Sleeping0x0FF66C7C0xCFEF7CE0ttyS1getty 0xCFE357F0974Sleeping0x0FF4B85C0xCF6EBCE0(none)in.telnetd 0xCFFDE470979Sleeping0x0FEB69500xCF675DB0ttyp0bash 0xCFFDEBB0982<Running0x0FF6EB6C0xCF7C3870ttyp0sync (gdb) The bulk of the work done by this ps macro is performed by the task_struct_show macro. As shown in Listing 14-13, the task_struct_show macro displays the following fields from each task_struct: •AddressAddressofthetask_structfortheprocess •PIDProcessID •StateCurrentstateoftheprocess •User_NIPUserspaceNextInstructionPointer •Kernel_SPKernelStackPointer •deviceDeviceassociatedwiththisprocess •commNameoftheprocess(orcommand) Itisrelativelyeasytomodifythemacrotoshowtheitemsofinterestforyourparticularkernel debugging task. The only complexity is in the simplicity of the macro language. Because function equivalentssuchasstrlendonotexistingdb'suser-definedcommandlanguage,screenformatting mustbedonebyhand. Listing14-14reproducesthetask_struct_showmacrothatproducedthepreviouslisting. Listing14-14.gdbtask_struct_showMacro 1definetask_struct_show 2#task_structaddrandPID 3printf"0x%08X%5d",$arg0,$arg0->pid 4 5#Placea'<'markeronthecurrenttask 6#if($arg0==current) 7#ForPowerPC,registerr2pointstothe"current"task 8if($arg0==$r2) 9printf"<" 10else 11printf"" 12end 13 14#State 15if($arg0->state==0) 16printf"Running" 17else 18if($arg0->state==1) 19printf"Sleeping" 20else 21if($arg0->state==2) 22printf"Disksleep" 23else 24if($arg0->state==4) 25printf"Zombie" 26else 27if($arg0->state==8) 28printf"sTopped" 29else 30if($arg0->state==16) 31printf"Wpaging" 32else 33printf"%2d",$arg0->state 34end 35end 36end 37end 38end 39end 40 41#UserNIP 42if($arg0->thread.regs) 43printf"0x%08X",$arg0->thread.regs->nip 44else 45printf"" 46end 47 48#Displaythekernelstackpointer 49printf"0x%08X",$arg0->thread.ksp 50 51#device 52if($arg0->signal->tty) 53printf"%s",$arg0->signal->tty->name 54else 55printf"(none)" 56end 57 58#comm 59printf"%s\n",$arg0->comm 60end Line3displaystheaddressofthetask_struct.Lines8through12displaytheprocessID.Ifthisis thecurrentprocess(theprocessthatwascurrentlyrunningonthisCPUatthetimethebreakpointwas hit),itismarkedwitha<character. Lines14through39decodeanddisplaythestateoftheprocess.Thisisfollowedbydisplaying theuserprocessnextinstructionpointer(NIP)andthekernelstackpointer(SP).Finally,thedevice associatedwiththeprocessisdisplayed,followedbythenameoftheprocess(storedinthe->comm elementofthetask_struct.) It is important to note that this macro is architecture dependent, as shown in lines 7 and 8. In general,macrossuchasthesearehighlyarchitecture-andversion-dependent.Anytimeachangein theunderlyingstructureismade,macrossuchasthesemustbeupdated.However,ifyouspendalot oftimedebuggingthekernelusinggdb,thepaybackisoftenworththeeffort. Forcompleteness,wepresentthefind_next_taskmacro.Itsimplementationislessthanobvious and deserves explanation. (It is assumed that you can easily deduce the task_struct_header that completestheseriesnecessaryforthepsmacropresentedinthissection.Itisnothingmorethana single line arranging the column headers with the correct amount of whitespace.) Listing 14-15 presentsthefind_next_taskmacrousedinourpsandfind_taskmacros. Listing14-15.gdbfind_next_taskMacro definefind_next_task #Givenataskaddress,findthenexttaskinthelinkedlist set$t=(structtask_struct*)$arg0 set$offset=((char*)&$t->tasks-(char*)$t) set$t=(structtask_struct*)((char*)$t->tasks.next-(char*)$offset) end The function performed by this macro is simple. The implementation is slightly less than straightforward.Thegoalistoreturnthe->nextpointer,whichpointstothenexttask_structonthe linked list. However, the task_struct structures are linked by the address of the struct list_head membercalledtasks,asopposedtothecommonpracticeofbeinglinkedbythestartingaddressof thetask_structitself.Becausethe->nextpointerpointstotheaddressofthetaskstructureelementin thenexttask_structonthelist,wemustsubtracttogettheaddressofthetopofthetask_structitself. Thevaluewesubtractfromthe->nextpointeristheoffsetfromthatpointer'saddresstothetopof task_struct. First we calculate the offset and then we use that offset to adjust the ->next pointer to pointtothetopoftask_struct.Figure14-5shouldmakethisclear. Figure14-5.Taskstructurelistlinking Now we present one final macro that will be useful in the next section when we discuss debugging loadable modules. Listing 14-16 is a simple macro that displays the kernel's list of currentlyinstalledloadablemodules. Listing14-16.gdbListModulesMacro 1definelsmod 2printf"Address\t\tModule\n" 3set$m=(structlist_head*)&modules 4set$done=0 5while(!$done) 6#list_headis4-bytesintostructmodule 7set$mp=(structmodule*)((char*)$m->next-(char*)4) 8printf"0x%08X\t%s\n",$mp,$mp->name 9if($mp->list->next==&modules) 10set$done=1 11end 12set$m=$m->next 13end 14end 15 16documentlsmod 17Listtheloadedkernelmodulesandtheirstartaddresses 18end This simple loop starts with the kernel's global variable module. This variable is a struct list_headthatmarksthestartofthelinkedlistofloadablemodules.Theonlycomplexityisthesame asthatdescribedinListing14-15.Wemustsubtractanoffsetfromthestructlist_headpointertopoint tothetopofthestructmodule.Thisisperformedinline7.Thismacroproducesasimplelistingof modulescontainingtheaddressofthestructmoduleandthemodule'sname.Hereisanexampleofits use: (gdb)lsmod AddressModule 0xD1012A80ip_conntrack_tftp 0xD10105A0ip_conntrack 0xD102F9A0loop (gdb)helplsmod Listtheloadedkernelmodulesandtheirstartaddresses (gdb) Macrossuchastheonespresentedhereareverypowerfuldebuggingaids.Youcancreatemacros inasimilarfashiontodisplayanythinginthekernelthatlendsitselftoeasyaccess,especiallythe majordatastructuresmaintainedaslinkedlists.Examplesincludeprocessmemorymapinformation, module information, file system information, and timer lists and so on. The information presented hereshouldgetyoustarted. 14.3.5.DebuggingLoadableModules ThemostcommonreasonforusingKGDBistodebugloadablekernelmodules,thatis,device drivers.Oneofthemoreconvenientfeaturesofloadablemodulesisthat,undermostcircumstances, it is not necessary to reboot the kernel for each new debugging session. You can start a debugging session, make some changes, recompile, and reload the module without the hassle and delay of a completekernelreboot. The complication associated with debugging loadable modules is in gaining access to the symbolic debug information contained in the module's object file. Because loadable modules are dynamicallylinkedwhentheyareloadedintothekernel,thesymbolicinformationcontainedinthe objectfileisuselessuntilthesymboltableisadjusted. Recallfromourearlierexampleshowweinvokegdbforakerneldebuggingsession: $ppc_4xx-gdbvmlinux Thislaunchesagdbdebuggingsessiononyourhost,andreadsthesymbolinformationfromthe LinuxkernelELFfilevmlinux.Ofcourse,youwillnotfindsymbolsforanyloadablemodulesinthis file.LoadablemodulesareseparatecompilationunitsandarelinkedasindividualstandaloneELF objects. Therefore, if we intend to perform any source-level debugging on a loadable module, we needtoloaditsdebugsymbolsfromtheELFfile.gdbprovidesthiscapabilityinitsadd-symbol-file command. The add-symbol-file command loads symbols from the specified object file, assuming that the moduleitselfhasalreadybeenloaded.However,wearefacedwiththechicken-and-eggsyndrome. Wedon'thaveanysymbolinformationuntiltheloadablemodulehasbeenloadedintothekerneland theadd-symbol-filecommandisissuedtoreadinthemodule'ssymbolinformation.However,after themodulehasbeenloaded,itistoolatetosetbreakpointsanddebugthemodule's*_initandrelated functionsbecausetheyhavealreadyexecuted. The solution tothisdilemma istoplaceabreakpointinthekernelcodethatisresponsiblefor loadingthemodule,afterithasbeenlinkedbutbeforeitsinitializationfunctionhasbeencalled.This workisdoneby.../kernel/module.c.Listing14-17reproducestherelevantportionsofmodule.c. Listing14-17.module.c:ModuleInitialization ... 1901down(¬ify_mutex); 1902notifier_call_chain(&module_notify_list,MODULE_STATE_COMING,mod); 1903up(¬ify_mutex); 1904 1905/*Startthemodule*/ 1906if(mod->init!=NULL) 1907ret=mod->init(); 1908if(ret<0){ 1909/*Initroutinefailed:abort.Trytoprotectusfrom 1910buggyrefcounters.*/ 1911mod->state=MODULE_STATE_GOING; ... Weloadthemoduleusingthemodprobeutility,whichwasdemonstratedinListing8-5inChapter 8,"DeviceDriverBasics,"andlookslikethis: $modprobeloop Thiscommandissuesaspecialsystemcallthatdirectsthekerneltoloadthemodule.Themodule loading begins at sys_init_module() in module.c. After the module has been loaded into kernel memory and dynamically linked, control is passed to the module's _init function. This is shown in lines 1906 and 1907 of Listing 14-17. We place our breakpoint here. This enables us to add the symbolfiletogdbandsubsequentlysetbreakpointsinthemodule.Wedemonstratethisprocessusing the Linux kernel's loopback driver called loop.ko. This module has no dependencies on other modulesandisreasonablyeasytodemonstrate. Listing14-18showsthegdbcommandstoinitiatethisdebuggingsessiononloop.ko. Listing14-18.InitiateModuleDebugSession:loop.ko 1$ppc-linux-gdb--silentvmlinux 2(gdb)connect 3breakinst()atarch/ppc/kernel/ppc-stub.c:825 4825} 5Breakpoint1at0xc0016b18:filekernel/panic.c,line74. 6Breakpoint2at0xc005a8c8:filefs/buffer.c,line296. 7(gdb)bmodule.c:1907 8Breakpoint3at0xc003430c:filekernel/module.c,line1907. 9(gdb)c 10Continuing. 11>>>>Hereweletthekernelfinishbooting 12andthenloadtheloop.komoduleonthetarget 13 14Breakpoint3,sys_init_module(umod=0x30029000,len=0x2473e, 15uargs=0x10016338"")atkernel/module.c:1907 161907ret=mod->init(); 17(gdb)lsmod 18AddressModule 190xD102F9A0loop 20(gdb)set$m=(structmodule*)0xD102F9A0. 21(gdb)p$m->module_core 22$1=(void*)0xd102c000 23(gdb)add-symbol-file./drivers/block/loop.ko0xd102c000 24addsymboltablefromfile"./drivers/block/loop.ko"at 25.text_addr=0xd102c000 26(yorn)y 27Readingsymbolsfrom/home/chris/sandbox/linux-2.6.13-amcc/drivers/block/loop.ko...done. Startingwithline2,weusethegdbuser-definedmacroconnectcreatedearlierinListing14-10 toconnecttothetargetboardandsetourinitialbreakpoints.Wethenaddthebreakpointinmodule.c, as shown in line 7, and we issue the continue command (c). Now the kernel completes the boot processandwe establishatelnetsession intothe target andloadtheloop.komodule(notshown). When the loopback module is loaded, we immediately hit breakpoint #3. gdb then displays the informationshowninlines14through16. Atthispoint,weneedtodiscovertheaddresswheretheLinuxkernellinkedourmodule's.text section. Linux stores this address in the module information structure struct module in the module_coreelement.UsingthelsmodmacrowedefinedinListing14-16,weobtaintheaddressof thestructmoduleassociatedwithourloop.komodule.Thisisshowninlines17through19.Nowwe use this structure address to obtain the module's .text address from the module_core structure member. We pass this address to the gdb add-symbol-file command, and gdb uses this address to adjustitsinternalsymboltabletomatchtheactualaddresseswherethemodulewaslinkedintothe kernel.Fromthere,wecanproceedintheusualmannertosetbreakpointsinthemodule,stepthrough code,examinedata,andsoon. Weconcludethissectionwithademonstrationofplacingabreakpointintheloopbackmodule's initializationfunctionsothatwecanstepthroughthemodule'sinitializationcode.Thecomplication here is that the kernel loads the module's initialization code into a separately allocated portion of memory so that it can be freed after use. Recall from Chapter 5, "Kernel Initialization," our discussionofthe__initmacro.Thismacroexpandsintoacompilerattributethatdirectsthelinkerto place the marked portion of code into a specially named ELF section. In essence, any function definedwiththisattributeisplacedinaseparateELFsectionnamed.init.text.Itsuseissimilartothe following: staticint__initloop_init(void){...} This invocation would place the compiled loop_init() function into the .init.text section of the loop.koobjectmodule.Whenthemoduleisloaded,thekernelallocatesachunkofmemoryforthe mainbodyofthemodule,whichispointedtobythestructmodulemembernamedmodule_core.It then allocates a separate chunk of memory to hold the .init.text section. After the initialization functioniscalled,thekernelfreesthememorythatcontainedtheinitializationfunction.Becausethe objectmoduleissplitlikethis,weneedtoinformgdbofthisaddressingschemetobeabletouse symbolicdatafordebuggingtheinitializationfunction.[97]Listing14-19demonstratesthesesteps. Listing14-19.DebuggingModuleinitCode $ppc_4xx-gdb-slientvmlinux (gdb)targetremote/dev/ttyS0 Remotedebuggingusing/dev/ttyS0 breakinst()atarch/ppc/kernel/ppc-stub.c:825 825} <<Placeabreakpointbeforecallingmoduleinit>> (gdb)bmodule.c:1907 Breakpoint1at0xc0036418:filekernel/module.c,line1907. (gdb)c Continuing. Breakpoint1,sys_init_module(umod=0xd102ef40,len=0x23cb3,uargs=0x10016338"") atkernel/module.c:1907 1907ret=mod->init(); <<Discoverinitaddressingfromstructmodule>> (gdb)lsmod AddressModule 0xD102EF40loop (gdb)set$m=(structmodule*)0xD102EF40 (gdb)p$m->module_core $1=(void*)0xd102b000 (gdb)p$m->module_init $2=(void*)0xd1031000 <<Nowloadasymbolfileusingthecoreandinitaddrs>> (gdb)add-symbol-file./drivers/block/loop.ko0xd102b000-s.init.text0xd1031000 addsymboltablefromfile"./drivers/block/loop.ko"at .text_addr=0xd102b000 .init.text_addr=0xd1031000 (yorn)y Readingsymbolsfrom/home/chris/sandbox/linux-2.6.13-amcc/drivers/block/loop.ko...done. (gdb)bloop_init Breakpoint3at0xd1031000:filedrivers/block/loop.c,line1244. (gdb)c Continuing. <<Breakpointhit,proceedtodebugmoduleinitfunction>> Breakpoint3,0xd1031000inloop_init()filedrivers/block/loop.c,line1244 1244if(max_loop<1||max_loop>256) (gdb) 14.3.6.printkDebugging Debugging kernel and device driver code using printk is a popular technique, mostly because printkhasevolvedintoaveryrobustmethod.Youcancallprintkfromalmostanycontext,including frominterrupthandlers.printkisthekernel'sversionofthefamiliarprintf()Clibraryfunction.printk isdefinedin.../kernel/printk.c. Itisimportanttounderstandthelimitationsofusingprintkfordebugging.First,printkrequiresa console device. Moreover, although the console device is configured as early as possible during kernelinitialization,therearemanycallstoprintkbeforetheconsoledevicehasbeeninitialized.We presentamethodtocopewiththislimitationlater,inSection14.5,"WhenItDoesn'tBoot." Theprintkfunctionallowstheadditionofastringmarkerthatidentifiesthelevelofseverityofa givenmessage.Theheaderfile.../include/linux/kernel.hdefineseightlevels: #defineKERN_EMERG"<0>"/*systemisunusable*/ #defineKERN_ALERT"<1>"/*actionmustbetakenimmediately*/ #defineKERN_CRIT"<2>"/*criticalconditions*/ #defineKERN_ERR"<3>"/*errorconditions*/ #defineKERN_WARNING"<4>"/*warningconditions*/ #defineKERN_NOTICE"<5>"/*normalbutsignificantcondition*/ #defineKERN_INFO"<6>"/*informational*/ #defineKERN_DEBUG"<7>"/*debug-levelmessages*/ Asimpleprintkmessagemightlooklikethis: printk("foo()enteredw/%s\n",arg); If the severity string is omitted, the kernel assigns a default severity level, which is defined in printk.c.Inrecentkernels,thisissetatseveritylevel4,KERN_WARNING.Specifyingprintkwitha severitylevelmightlooksomethinglikethis: printk(KERN_CRIT"vmallocfailedinfoo()\n"); Bydefault,allprintkmessagesbelowapredefinedloglevelaredisplayedonthesystemconsole device.Thedefaultloglevelisdefinedinprintk.c.InrecentLinuxkernels,ithasthevalue7.This meansthatanyprintkmessagethatisgreaterinimportancethanKERN_DEBUGwillbedisplayedon theconsole. You can set the default kernel loglevel in a variety of ways. At boot time, you can specify the defaultloglevelonyourtargetboardbypassingtheappropriatekernelcommandlineparametersto the kernel at boot time. Three kernel command line options defined in main.c affect the default loglevel: •debugSetstheconsoleloglevelto10 •quietSetstheconsoleloglevelto4 •loglevel=Setstheconsolelogleveltoyourchoiceofvalue Usingdebugeffectivelydisplayseveryprintkmessage.Usingquietdisplaysallprintkmessages ofseverityKERN_ERRorhigher. printkmessagescanbeloggedtofilesonyourtargetorviathenetwork.Useklogd(kernellog daemon)andsyslogd(systemlogdaemon)tocontroltheloggingbehaviorofprintkmessages.These popularutilitiesaredescribedinmanpagesandmanyLinuxreferences,andarenotdescribedhere. 14.3.7.MagicSysReqKey This useful debugging aid is invoked through a series of special predefined key sequences that send messages directly to the kernel. For many target architectures and boards, you use a simple terminalemulatoronaserialportasasystemconsole.Forthesearchitectures,theMagicSysReqkey isdefinedasabreakcharacterfollowedbyacommandcharacter.Consultthedocumentationonthe terminalemulatoryouuseforhowtosendabreakcharacter.ManyLinuxdevelopersusetheminicom terminalemulator.Forminicom,thebreakcharacterissentbytypingCtl-AF.Aftersendingthebreak inthismanner,youhave5secondstoenterthecommandcharacterbeforethecommandtimesout. Thisusefulkerneltoolcanbeveryhelpfulfordevelopmentanddebugging,butitcanalsocause datalossandsystemcorruption.Indeed,thebcommandimmediatelyrebootsyoursystemwithoutany notificationorpreparation.Openfilesarenotclosed,disksarenotsynced,andfilesystemsarenot unmounted.Whenthereboot(b)commandisissued,controlisimmediatelypassedtotheresetvector ofyourarchitectureinamostabruptandstunningmanner.Usethispowerfultoolatyourownperil! ThisfeatureiswelldocumentedintheLinuxkerneldocumentationsubdirectoryinafilecalled sysrq.txt. There you find the details for many architectures and the description of available commands. For example, another way to set the kernel loglevel just discussed is to use the Magic SysReq key.Thecommandisanumberfrom0through9,whichresultsinthedefaultloglevelbeingsettothe numberofthecommand.Fromminicom,pressCtl-AFfollowedbyanumber,suchas9.Hereishow itlooksontheterminal: $SysRq:ChangingLoglevel Loglevelsetto9 Commandscanbeusedtodumpregisters,shutdownyoursystem,rebootyoursystem,dumpalist of processes, dump current memory information to your console, and more. See the documentation fileinanyrecentLinuxkernelforthedetails. This feature is most commonly used when something causes your system to lock up. Often the MagicSysReqkeyprovidesawaytolearnsomethingfromanotherwisedeadsystem. 14.4.Hardware-AssistedDebugging By now you've probably realized that you cannot debug very early kernel-startup code with KGDB. This is because KGDB is not initialized until after most of the low-level hardwareinitializationcodehasexecuted.Furthermore,ifyouareassignedthetaskofbringingupabrand-new board design and porting a bootloader and the Linux kernel, having a hardware-debug probe is without a doubt the most efficient means of debugging problems in these early stages of board porting. Youcanchoosefromawidevarietyofhardware-debugprobes.Fortheexamplesinthissection, weuseaunitmanufacturedbyAbatroncalledtheBDI-2000(seewww.abatron.ch).Theseunitsare often called JTAG probes because they use a low-level communications method that was first employed for boundary scan testing of integrated circuits defined by the Joint Test Action Group (JTAG). AJTAGprobecontainsasmallconnectordesignedforconnectiontoyourtargetboard.Itisoften a simple square-pin header and ribbon cable arrangement. Most modern high-performance CPUs containaJTAGinterfacethatisdesignedtoprovidethissoftwaredebuggingcapability.TheJTAG probeconnectstothisCPUJTAGinterface.TheothersideoftheJTAGprobeconnectstoyourhost developmentsystemusuallyviaEthernet,USB,oraparallelport.Figure14-6detailsthesetupfor theAbatronunit. Figure14-6.HardwareJTAGprobedebugging JTAGprobescanbecomplicatedtosetup.ThisisadirectresultofthecomplexityoftheCPUto which it is connected. When power is applied to a target board and its CPU comes out of reset, almostnothingisinitialized.Infact,manyprocessorsneedatleastasmallamountofinitialization beforetheycandoanything.Manymethodsareavailableforgettingthisinitialconfigurationintothe CPU.SomeCPUsreadahardware-configurationwordorinitialvaluesofspecificpinstolearntheir power-on configuration. Others rely on reading a default location in a simple nonvolatile storage devicesuchasFlash.WhenusingaJTAGprobe,especiallyforbringingupanewboarddesign,a minimumlevelofCPUandboardinitializationmustbeperformedbeforeanythingelsecanbedone. ManyJTAGprobesrelyonaconfigurationfileforthisinitialization. TheAbatronunitusesaconfigurationfiletoinitializethetargethardwareitisconnectedto,as well as to define other operational parameters of the debugger. This configuration file contains directivesthatinitializetheCPU,memorysystem,andothernecessaryboard-levelhardware.Itisthe developer'sresponsibilitytocustomizethisconfigurationfilewiththeproperdirectivesforhisown board. The details on the configuration command syntax can be found in the JTAG probe's documentation. However, only the embedded developer can create the unique configuration file required for a given board design. This requires detailed knowledge of the CPU and board-level design features. Much like creating a custom Linux port for a new board, there is no shortcut or substituteforthistask. Appendix F, "Sample BDI-2000 Configuration File," contains a sample Abatron configuration fileforacustomboardbasedontheFreescaleSemiconductorMPC5200embeddedcontroller.Inthat appendix, you can see the necessary setup for a custom board. Notice the liberal use of comments describing various registers and initialization details. This makes it easier to update and maintain overtime,anditcanhelpyoutogetitrightthefirsttime. Hardwareprobesaregenerallyusedintwoways.Mosthaveauserinterfaceofsometypethat enables the developer to use features of the probe. Examples of this are to program Flash or downloadbinaryimages.Thesecondusageisasafrontendtogdborothersource-leveldebuggers. Wedemonstratebothusagescenarios. 14.4.1.ProgrammingFlashUsingaJTAGProbe ManyhardwareprobesincludethecapabilitytoprogramawidevarietyofFlashmemorychips. TheAbatronBDI-2000isnoexception.TheBDI-2000configurationfileincludesa[FLASH]section to define the characteristics of the target Flash. Refer to Appendix F for a sample. The [FLASH] sectiondefinesattributesoftheFlashchipasusedinaparticulardesign,suchasthechiptype,the sizeofthedevice,anditsdatabuswidth.Alsodefinedarethelocationinmemoryandsomewayto describethechip'sstorageorganization. WhenupdatingoneportionoftheFlash,youoftenwanttopreservethecontentsofotherportions ofthesameFlash.Inthiscase,yourhardwareprobemusthavesomewaytolimitthesectorsthatare erased.InthecaseoftheAbatronunit,thisisdonebyaddingalinestartingwiththekeywordERASE foreachsectortobeerased.WhentheerasecommandisissuedtotheAbatronunitviaitstelnetuser interface, all sectors defined with an ERASE specification are erased. Listing 14-20 demonstrates erasingaportionofFlashonatargetboardandsubsequentlyprogramminganewU-Bootbootloader image. Listing14-20.EraseandProgramFlash $telnetbdi Trying192.168.1.129... Connectedtobdi(192.168.1.129). Escapecharacteris'^]'. BDIDebuggerforEmbeddedPowerPC ================================= ...(largevolumeofhelptext) uei>erase Erasingflashat0xfff00000 Erasingflashat0xfff10000 Erasingflashat0xfff20000 Erasingflashat0xfff30000 Erasingflashat0xfff40000 Erasingflashpassed uei>prog0xfff00000u-boot.binBIN Programmingu-boot.bin,pleasewait.... Programmingflashpassed uei> First we establish a telnet session to the Abatron BDI-2000. After some initialization, we are presentedwithacommandprompt.Whentheerasecommandisissued,theAbatrondisplaysalineof outputforeachsectiondefinedintheconfigurationfile.WiththeconfigurationshowninAppendixF, wedefinedfiveerasesectors.Thisreservesupto256KBofspacefortheU-Bootbootloaderbinary. Theprogcommandisshownwithallthreeofitsoptionalparameters.Thesespecifythelocation in memorywhere the newimage is to beloaded,thenameoftheimagefile,andthe format ofthe fileinthiscase,abinaryfile.YoucanspecifytheseparametersintheBDI-2000configurationfile.In thiscase,thecommandreducestosimplyprogwithoutparameters. This example only scratches the surface of these two BDI-2000 commands. Many more combinationsofusageandcapabilitiesaresupported.EachhardwareJTAGprobehasitsownway tospecifyFlasherasureandprogrammingcapabilities.Consultthedocumentationforyourparticular deviceforthespecifics. 14.4.2.DebuggingwithaJTAGProbe InsteadofinterfacingdirectlywithaJTAGprobeviaitsuserinterface,manyJTAGprobescan interfacewithyoursource-leveldebugger.Byfarthemostpopulardebuggersupportedbyhardware probesisthegdbdebugger.Inthisusagescenario,gdbisinstructedtobeginadebugsessionwiththe targetviaanexternalconnection,usuallyanEthernetconnection.Ratherthancommunicatedirectly with the JTAG probe via a user interface, the debugger passes commands back and forth between itselfandtheJTAGprobe.Inthismodel,theJTAGprobeusesthegdbremoteprotocoltocontrolthe hardwareonbehalfofthedebugger.ReferagaintoFigure14-6forconnectiondetails. JTAG probes are especially useful for source-level debugging of bootloader and early startup code. In this example, we demonstrate the use of gdb and an Abatron BDI-2000 for debugging portionsoftheU-BootbootloaderonaPowerPCtargetboard. Manyprocessorscontaindebuggingregistersthatincludethecapabilitytosettraditionaladdress breakpoints(stopwhentheprogramreachesaspecificaddress)aswellasdatabreakpoints(stopon conditional access of a specified memory address). When debugging code resident in read-only memorysuchasFlash,thisistheonlywaytosetabreakpoint.However,theseregistersaretypically limited.Manyprocessorscontainonlyoneortwosuchregisters.Thislimitationmustbeunderstood beforeusinghardwarebreakpoints.Thefollowingexampledemonstratesthis. UsingasetupsuchasthatshowninFigure14-6,assumethatourtargetboardhasU-Bootstored in Flash. When we presented bootloaders in Chapter 7, you learned that U-Boot and other bootloaderstypicallycopythemselvesintoRAMassoonaspossibleafterstartup.Thisisbecause hardwareread(andwrite)cyclesfromRAMareordersofmagnitudefasterthantypicalread-only memory devices such as Flash. This presents two specific debugging challenges. First, we cannot modify the contents of read-only memory (to insert a software breakpoint), so we must rely on processor-supportedbreakpointregistersforthispurpose. Thesecondchallengecomesfromthefactthatonlyoneoftheexecutioncontexts(FlashorRAM) can be represented by the ELF executable file from which gdb reads its symbolic debugging information.InthecaseofU-Boot,itislinkedfortheFlashenvironmentwhereitisinitiallystored. Theearlycoderelocatesitselfandperformsanynecessaryaddressadjustments.Thismeansthatwe needtoworkwithgdbwithinbothoftheseexecutioncontexts.Listing14-21showsanexampleof suchadebugsession. Listing14-21.U-BootDebuggingUsingJTAGProbe $ppc-linux-gdb--silentu-boot (gdb)targetremotebdi:2001 Remotedebuggingusingbdi:2001 _start()at/home/chris/sandbox/u-boot-1.1.4/cpu/mpc5xxx/start.S:91 91lir21,BOOTFLAG_COLD/*NormalPower-On*/ Currentlanguage:auto;currentlyasm <<Debugaflashresidentcodesnippet>> (gdb)monbreakhard (gdb)bboard_init_f Breakpoint1at0xfff0457c:fileboard.c,line366. (gdb)c Continuing. Breakpoint1,board_init_f(bootflag=0x7fc3afc)atboard.c:366 366gd=(gd_t*)(CFG_INIT_RAM_ADDR+CFG_GBL_DATA_OFFSET); Currentlanguage:auto;currentlyc (gdb)bt #0board_init_f(bootflag=0x1)atboard.c:366 #10xfff0456cinboard_init_f(bootflag=0x1)atboard.c:353 (gdb)iframe Stacklevel0,frameat0xf000bf50: pc=0xfff0457cinboard_init_f(board.c:366);savedpc0xfff0456c calledbyframeat0xf000bf78 sourcelanguagec. Arglistat0xf000bf50,args:bootflag=0x1 Localsat0xf000bf50,Previousframe'sspis0x0 <<Nowdebugamemoryresidentcodesnippetafterrelocation>> (gdb)del1 (gdb)symbol-file Discardsymboltablefrom'/home/chris/sandbox/u-boot-1.1.4-powerdna/u-boot'? (yorn)y Nosymbolfilenow. (gdb)add-symbol-fileu-boot0x7fa8000 addsymboltablefromfile"u-boot"at .text_addr=0x7fa8000 (yorn)y Readingsymbolsfromu-boot...done. (gdb)bboard_init_r Breakpoint2at0x7fac6c0:fileboard.c,line608. (gdb)c Continuing. Breakpoint2,board_init_r(id=0x7f85f84,dest_addr=0x7f85f84)atboard.c:608 608gd=id;/*initializeRAMversionofglobaldata*/ (gdb)iframe Stacklevel0,frameat0x7f85f38: pc=0x7fac6c0inboard_init_r(board.c:608);savedpc0x7fac6b0 calledbyframeat0x7f85f68 sourcelanguagec. Arglistat0x7f85f38,args:id=0x7f85f84,dest_addr=0x7f85f84 Localsat0x7f85f38,Previousframe'sspis0x0 (gdb)monbreaksoft (gdb) Studythisexamplecarefully.Somesubtletiesaredefinitelyworthtakingthetimetounderstand. First,weconnecttotheAbatronBDI-2000usingthetargetremotecommand.TheIPaddressinthis case is that of the Abatron unit, represented by the symbolic name bdi.[98] The Abatron BDI-2000 usesport2001foritsremotegdbprotocolconnection. NextweissueacommandtotheBDI-2000usingthegdbmoncommand.Themoncommandtells gdbtopasstherestofthecommanddirectlytotheremotehardwaredevice.Therefore,monbreak hardsetstheBDI-2000intohardwarebreakpointmode. We then set a hardware breakpoint at board_init_f. This is a routine that executes while still running out of Flash memory at address 0xfff0457c. After the breakpoint is defined, we issue the continueccommandtoresumeexecution.Immediately,thebreakpointatboard_init_fisencountered, andwearefreetodotheusualdebuggingactivities,includingsteppingthroughcodeandexamining data.Youcanseethatwehaveissuedthebtcommandtoexaminethestackbacktraceandtheiframe commandtoexaminethedetailsofthecurrentstackframe. Nowwecontinueexecutionagain,butthistimeweknowthatU-BootcopiesitselftoRAMand resumesexecutionfromitscopyinRAM.Soweneedtochangethedebuggingcontextwhilekeeping the debugging session alive. To accomplish this, we discard the current symbol table (symbol-file command with no arguments) and load in the same symbol file again using the add-symbol-file command.Thistime,weinstructgdbtooffsetthesymboltabletomatchwhereU-Boothasrelocated itself to memory. This ensures that our source code and symbolic debugging information match the actualmemoryresidentimage. Afterthenewsymboltableisloaded,wecanaddabreakpointtoalocationthatweknowwill reside in RAM when it is executed. This is where one of the subtle complications is exposed. BecauseweknowthatU-BootiscurrentlyrunninginFlashbutisabouttomoveitselftoRAMand jumptoitsRAM-basedcopy,wemuststilluseahardwarebreakpoint.Considerwhathappensatthis pointifweuseasoftwarebreakpoint.gdbdutifullywritesthebreakpointopcodeintothespecified memorylocation,butU-BootoverwritesitwhenitcopiesitselftoRAM.Thenetresultisthatthe breakpointisneverhit,andwebegintosuspectthatourtoolsarebroken.AfterU-Boothasentered theRAMcopyandoursymboltablehasbeenupdatedtoreflecttheRAM-basedaddresses,weare freetouseRAM-basedbreakpoints.ThisisreflectedbythelastcommandinListing14-21settingthe Abatronunitbacktosoftbreakpointmode. Why do we care about using hardware versus software breakpoints? If we had unlimited hardware breakpoint registers, we wouldn't. But this is never the case. Here is what it looks like whenyourunoutofprocessor-supportedhardwarebreakpointregistersduringadebugsession: (gdb)bflash_init Breakpoint3at0x7fbebe0:fileflash.c,line70. (gdb)c Continuing. warning:Cannotinsertbreakpoint3: Erroraccessingmemoryaddress0x7fbebe0:Unknownerror4294967295. Becausewearedebuggingremotely,wearen'ttoldabouttheresourceconstraintuntilwetryto resume after entering additional breakpoints. This is because of the way gdb handles breakpoints. Whenabreakpointishit,gdbrestoresallthebreakpointswiththeoriginalopcodesforthatparticular memory location. When it resumes execution, it restores the breakpoint opcodes at the specified locations.Youcanobservethisbehaviorbyenablinggdb'sremotedebugmode: (gdb)setdebugremote1 14.5.WhenItDoesn'tBoot OneofthemostfrequentlyaskedquestionsonthevariousmailingliststhatserveembeddedLinux goessomethinglikethis: IamtryingtobootLinuxonmyboard,andIgetstuckafterthismessageprintstomyconsole: "UncompressingKernelImage...OK." Thus starts the long and sometimes frustrating learning curve of embedded Linux! Many things thatcangowrongcouldleadtothiscommonfailure.WithsomeknowledgeandaJTAGdebugger, therearewaystodeterminewhatwentawry. 14.5.1.EarlySerialDebugOutput ThefirsttoolyoumighthaveavailableisCONFIG_SERIAL_TEXT_DEBUG.ThisLinuxkernelconfigurationoptionaddssupportfordebugmessagesveryearlyinthebootprocess.Atthepresent time,thisfeatureislimitedtothePowerPCarchitecture,butnothingpreventsyoufromduplicating thefunctionalityinotherarchitectures.Listing14-22providesanexampleofthisfeatureinuseona PowerPCtargetusingtheU-Bootbootloader. Listing14-22.EarlySerialTextDebug ##Bootingimageat00200000... ImageName:Linux-2.6.14 Created:2005-12-1922:24:03UTC ImageType:PowerPCLinuxKernelImage(gzipcompressed) DataSize:607149Bytes=592.9kB LoadAddress:00000000 EntryPoint:00000000 VerifyingChecksum...OK UncompressingKernelImage...OK idmach():done<==Startofmessagesenabledby MMU:enter<==CONFIG_SERIAL_TEXT_DEBUG MMU:hwinit MMU:mapin MMU:setio MMU:exit setup_arch:enter setup_arch:bootmem arch:exit arch:realexit Usingthisfeature,youcanoftentellwhereyourboardisgettingstuckduringthebootprocess.Of course,youcanaddyourownearlydebugmessagesinotherplacesinthekernel.Hereisanexample ofitsusagefoundin.../arch/ppc/mm/init.c: /*MapinallofRAMstartingatKERNELBASE*/ if(ppc_md.progress) ppc_md.progress("MMU:mapin",0x301); mapin_ram(); The AMCC Yosemite platform is an excellent example of this infrastructure. Consult the followingfilesintheLinuxsourcetree[99]fordetailsofhowthisdebuggingsystemisimplemented: File Function Purpose Serialportsetup,calledbyyosemite.cplatformgen550_dbg.c gen550_init initializationfile gen550_dbg.c gen550_progress Low-levelserialoutputroutine Bindsplatform-specificprogressroutinetogenericppc ibm44x_common.c ibm44x_platform_init machine-dependentinfrastructure 14.5.2.DumpingtheprintkLogBuffer WhenwediscussedprintkdebugginginSection14.3.6,wepointedoutsomeofthelimitationsof thismethod.printkitselfisaveryrobustimplementation.Oneofitsshortcomingsisthatyoucan'tsee any printk messages until later in the boot sequence when the console device has been initialized. Veryoften,whenyourboardhangsonboot,quiteafewmessagesarestuckintheprintkbuffer.Ifyou knowwheretofindthem,youcanoftenpinpointtheexactproblemthatiscausingtheboottohang. Indeed, many times you will discover that the kernel has encountered an error that led to a call to panic(). The output from panic() has likely been dumped into the printk buffer, and you can often pinpointtheexactlineofoffendingcode. ThisisbestaccomplishedwithaJTAGdebugger,butitisstillpossibletouseabootloaderand itsmemorydumpcapabilitytodisplaythecontentsoftheprintkbufferafterareset.Somecorruption ofmemorycontentsmightoccurasaresultofthereset,butlogbuffertextisusuallyveryreadable. The actual buffer where printk stores its message text is declared in the printk source file .../kernel/printk.c. staticchar__log_buf[__LOG_BUF_LEN]; We can easily determine the linked location of this buffer from the Linux kernel map file System.map. $grep__log_bufSystem.map c022e5a4b__log_buf Now if the system happens to hang upon booting, right after displaying the "Uncompressing Kernel Image . . . OK" message, reboot and use the bootloader to examine the buffer. Because the relationship between kernel virtual memory and physical memory is fixed and constant on a given architecture, we can do a simple conversion. The address of __log_buf shown earlier is a kernel virtualaddress;wemustconvertittoaphysicaladdress.OnthisparticularPowerPCarchitecture, thatconversionisasimplesubtractionoftheconstantKERNELBASEaddress,0xc0000000.Thisis whereweprobeinmemorytoreadthecontents,ifany,oftheprintklogbuffer. Listing14-23isanexampleofthelistingasdisplayedbytheU-Bootmemorydumpcommand. Listing14-23.DumpofRawprintkLogBuffer =>md22e5a4 0022e5a4:3c353e4c696e75782076657273696f6e<5>Linuxversion 0022e5b4:20322e362e313320286368726973406a2.6.13(chris@ 0022e5c4:756e696f722920286763632076657273junior)(gcc 0022e5d4:696f6e20332e342e3320284d6f6e7461version3.4.3(Monta 0022e5e4:566973746120332e342e332d32352e30Vista3.4.3-25.0 0022e5f4:2e37302e303530313936312032303035.70.05019612005 0022e604:2d31322d313829292023313120547565-12-18))#11Tue 0022e614:204665622031342032313a30353a3036Feb1421:05:06 0022e624:2045535420323030360a3c343e414d43EST2006.<4>AMC 0022e634:4320506f776572504320343430455020CPowerPC440EP 0022e644:596f73656d69746520506c6174666f72YosemitePlatform. 0022e654:6d0a3c373e4f6e206e6f646520302074<7>Onnode0 0022e664:6f74616c70616765733a203635353336totalpages:65536 0022e674:0a3c373e2020444d41207a6f6e653a20.<7>DMAzone: 0022e684:36353533362070616765732c204c494665536pages,LIF 0022e694:4f2062617463683a33310a3c373e2020Obatch:31.<7> => 0022e6a4:4e6f726d616c207a6f6e653a20302070Normalzone:0 0022e6b4:616765732c204c49464f206261746368pages,LIFObatch 0022e6c4:3a310a3c373e2020486967684d656d20:1.<7>HighMemzone: 0022e6d4:7a6f6e653a20302070616765732c204c0pages, 0022e6e4:49464f2062617463683a310a3c343e42LIFObatch:1.<4> 0022e6f4:75696c742031207a6f6e656c69737473Built1zonelists 0022e704:0a3c353e4b65726e656c20636f6d6d61.<5>Kernelcommand 0022e714:6e64206c696e653a20636f6e736f6c65line:console 0022e724:3d74747953302c31313532303020726f=ttyS0,115200 0022e734:6f743d2f6465762f6e66732072772069root=/dev/nfsrw 0022e744:703d646863700a3c343e504944206861ip=dhcp.<4>PID 0022e754:7368207461626c6520656e7472696573hashtableentries 0022e764:3a203230343820286f726465723a2031:2048(order: 0022e774:312c203332373638206279746573290a11,32768bytes). 0022e784:00000000000000000000000000000000................ 0022e794:00000000000000000000000000000000................ => It's not very pretty to read, but the data is there. We can see in this particular example that the kernel crashed someplace after initializing the PID hash table entries. With some additional use of printkmessages,wecanbegintocloseinontheactualsourceofthecrash. Asshowninthisexample,thisisatechniquethatcanbeusedwithnoadditionaltools.Youcan seetheimportanceofsomekindofearlyserialportoutputduringbootifyouareworkingonanew boardport. 14.5.3.KGDBonPanic IfKGDBisenabled,thekernelattemptstopasscontrolbacktoKGDBuponerrorexceptions.In somecases,theerroritselfwillbereadilyapparent.Tousethisfeature,aconnectionmustalreadybe established between KGDB and gdb. When the exception condition occurs, KGDB emits a Stop Replypackettogdb,indicatingthereasonforthetrapintothedebughandler,aswellastheaddress wherethetrapconditionoccurred.Listing14-24illustratesthesequence. Listing14-24.TrappingCrashonPanicUsingKGDB $ppc-_4xx-gdb--silentvmlinux (gdb)targetremote/dev/ttyS0 Remotedebuggingusing/dev/ttyS0 Malformedresponsetooffsetquery,qOffsets (gdb)targetremote/dev/ttyS0 Remotedebuggingusing/dev/ttyS0 breakinst()atarch/ppc/kernel/ppc-stub.c:825 825} (gdb)c Continuing. <<KGDBgainscontrolfrompanic()oncrash>> ProgramreceivedsignalSIGSEGV,Segmentationfault. 0xc0215d6cinpcibios_init()atarch/ppc/kernel/pci.c:1263 1263*(int*)-1=0; (gdb)bt #00xc0215d6cinpcibios_init()atarch/ppc/kernel/pci.c:1263 #10xc020e728indo_initcalls()atinit/main.c:563 #20xc020e7c4indo_basic_setup()atinit/main.c:605 #30xc0001374ininit(unused=0x20)atinit/main.c:677 #40xc00049d0inkernel_thread() Previousframeinnertothisframe(corruptstack?) (gdb) The crash in this example was contrived by a simple write to an invalid memory location (all ones).WefirstestablishaconnectionfromgdbtoKGDBandallowthekerneltocontinuetoboot. Notice that we didn't even bother to set breakpoints. When the crash occurs, we see the line of offendingcodeandgetanicebacktracetohelpusdetermineitscause. 14.6.ChapterSummary • Linux kernel debugging presents many complexities, especially in a cross-development environment. Understanding how to navigate these complexities is the key to successful kernel debugging. • KGDB is a very useful kernel-level gdb stub that enables direct symbolic source-level debugginginsidetheLinuxkernelanddevicedrivers.Itusesthegdbremoteprotocoltocommunicate toyourhost-basedcross-gdb. •Understanding(andminimizing)compileroptimizationshelpsmakesenseofseeminglystrange debuggerbehaviorwhensteppingthroughcompiler-optimizedcode. • gdb supports user-defined commands, which can be very useful for automating tedious debuggingtaskssuchasiteratingkernellinkedlistsandaccessingcomplexvariables. •Kernel-loadablemodulespresenttheirownchallengestosource-leveldebugging.Themodule's initialization routine can be debugged by placing a breakpoint in module.c at the call to module>init(). • printk and the Magic SysReq key provide additional tools to help isolate problems during kerneldevelopmentanddebugging. • Hardware-assisted debugging via a JTAG probe enables debugging Flash or ROM resident codewhereotherdebuggingmethodscanbecumbersomeorotherwiseimpossible. •EnablingCONFIG_SERIAL_TEXT_DEBUGonarchitectureswherethisfeatureissupportedis apowerfultoolfordebugginganewkernelport. •Examiningtheprintklog_bufoftenleadstothecauseofasilentkernelcrashonboot. •KGDBpassescontroltogdbonakernelpanic,enablingyoutoexamineabacktraceandisolate thecauseofthekernelpanic. 14.6.1.SuggestionsforAdditionalReading LinuxKernelDevelopment,2ndEdition RobertLove NovellPress,2005 TheLinuxKernelPrimer ClaudiaSalzbergRodriguezetal. PrenticeHall,2005 "UsingtheGNUCompilerCollection" Richard M. Stallman and the GCC Developer Community GNU Press, a division of Free SoftwareFoundation http://gcc.gnu.org/onlinedocs/ KGDBSourceforgehomepage http://sourceforge.net/projects/KGDB DebuggingwithGDB RichardStallman,RolandPesch,StanShebs,etal. FreeSoftwareFoundation www.gnu.org/software/gdb/documentation/ ToolInterfaceStandards DWARFDebuggingInformationFormatSpecification Version2.0 TISCommittee,May1995 Chapter15.DebuggingEmbeddedLinuxApplications Inthepreviouschapter,weexploredtheuseofGDBfordebuggingkernelcodeandcoderesident inFlash,suchasbootloadercode.Inthischapter,wecontinueourcoverageofGDBfordebugging application code in user space. We extend our coverage of remote debugging and the tools and techniquesusedforthispeculiardebuggingenvironment. 15.1.TargetDebugging We already explored several important debugging tools in Chapter 13, "Development Tools." strace and ltrace can be used to observe and characterize a process's behavior and often isolate problems. dmalloc can help isolate memory leaks and profile memory usage. ps and top are both usefulforexaminingthestateofprocesses.Theserelativelysmalltoolsaredesignedtorundirectly onthetargethardware. Debugging Linux application code on an embedded system has its own unique challenges. Resourcesonyourembeddedtargetareoftenlimited.RAMandnonvolatilestoragelimitationsmight prevent you from running target-based development tools. You might not have an Ethernet port or other high-speed connection. Your target embedded system might not have a graphical display, keyboard,ormouse. Thisiswhereyourcross-developmenttoolsandanNFSrootmountenvironmentcanyieldlarge dividends.Manytools,especiallyGDB,havebeenarchitectedtoexecuteonyourdevelopmenthost whileactuallydebuggingcodeonaremotetarget.GDBcanbeusedtointeractivelydebugyourtarget code or to perform a postmortem analysis of a core file generated by an application crash. We coveredthedetailsofapplicationcoredumpanalysisinChapter13. 15.2.Remote(Cross)Debugging Cross-development tools were developed primarily to overcome the resource limitations of embedded platforms. A modest-size application compiled with symbolic debug information can easily exceed several megabytes. With cross-debugging, the heavy lifting can be done on your developmenthost.Whenyouinvokeyourcross-versionofGDBonyourdevelopmenthost,youpass itanELFfilecompiledwithsymbolicdebuginformation.Onyourtarget,thereisnoreasonyoucan't strip[100]theELFfileofallunnecessarydebugginginfotokeeptheresultingimagetoitsminimum size. WeintroducedthereadelfutilityinChapter13.InChapter14,"KernelDebuggingTechniques," we used it to examine the debug information in an ELF file compiled with symbolic debugging information.Listing15-1containstheoutputofreadelfforarelativelysmallwebserverapplication compiledfortheARMarchitecture. Listing15-1.ELFFileDebugInfoforExampleProgram $xscale_be-readelf-Swebsdemo Thereare39sectionheaders,startingatoffset0x3dfd0: SectionHeaders: [Nr]NameTypeAddrOffSizeESFlgLkInfAl [0]NULL0000000000000000000000000 [1].interpPROGBITS0000815400015400001300A001 [2].note.ABI-tagNOTE0000816800016800002000A004 [3].note.numapolicyNOTE0000818800018800007400A004 [4].hashHASH000081fc0001fc00022c04A504 [5].dynsymDYNSYM0000842800042800046010A614 [6].dynstrSTRTAB0000888800088800021100A001 [7].gnu.versionVERSYM00008a9a000a9a00008c02A502 [8].gnu.version_rVERNEED00008b28000b2800002000A614 [9].rel.pltREL00008b48000b4800021808A5114 [10].initPROGBITS00008d60000d6000001800AX004 [11].pltPROGBITS00008d78000d7800033804AX004 [12].textPROGBITS000090b00010b0019fe400AX004 [13].finiPROGBITS0002309401b09400001800AX004 [14].rodataPROGBITS000230b001b0b00023d000A008 [15].ARM.extabPROGBITS0002548001d48000000000A001 [16].ARM.exidxARM_EXIDX0002548001d48000000800AL1204 [17].eh_frame_hdrPROGBITS0002548801d48800002c00A004 [18].eh_framePROGBITS000254b401d4b400007c00A004 [19].init_arrayINIT_ARRAY0002d53001d53000000400WA004 [20].fini_arrayFINI_ARRAY0002d53401d53400000400WA004 [21].jcrPROGBITS0002d53801d53800000400WA004 [22].dynamicDYNAMIC0002d53c01d53c0000d008WA604 [23].gotPROGBITS0002d60c01d60c00011804WA004 [24].dataPROGBITS0002d72801d7280003c000WA008 [25].bssNOBITS0002dae801dae80001c800WA004 [26].commentPROGBITS0000000001dae800094000001 [27].debug_arangesPROGBITS0000000001e4280004a000008 [28].debug_pubnamesPROGBITS0000000001e8c8001aae00001 [29].debug_infoPROGBITS00000000020376013d2700001 [30].debug_abbrevPROGBITS0000000003409d002ede00001 [31].debug_linePROGBITS00000000036f7b0034a200001 [32].debug_framePROGBITS0000000003a42000338000004 [33].debug_strPROGBITS0000000003d7a000067900001 [34].note.gnu.arm.ideNOTE0000000003de1900001c00001 [35].debug_rangesPROGBITS0000000003de3500001800001 [36].shstrtabSTRTAB0000000003de4d00018300001 [37].symtabSYMTAB0000000003e5e8004bd010387734 [38].strtabSTRTAB000000000431b80021bf00001 KeytoFlags: W(write),A(alloc),X(execute),M(merge),S(strings) I(info),L(linkorder),G(group),x(unknown) O(extraOSprocessingrequired)o(OSspecific),p(processorspecific) $ YoucanseefromListing15-1thattherearemanysectionscontainingdebuginformation.Thereis alsoa.commentsectionthatcontainsmorethan2KB(0x940)ofinformationthatisnotnecessaryfor theapplicationtofunction.Thesizeofthisexamplefile,includingdebuginformation,ismorethan 275KB. $ls-lwebsdemo -rwxrwxr-x1chrischris283511Nov818:48websdemo Ifwestripthisfileusingthestriputility,wecanminimizeitssizetopreserveresourcesonour targetsystem.Listing15-2showstheresults. Listing15-2.StripTargetApplication $xscale_be-strip-s-R.comment-owebsdemo-strippedwebsdemo $ls-lwebsdemo* -rwxrwxr-x1chrischris283491Apr909:19websdemo -rwxrwxr-x1chrischris123156Apr909:21websdemo-stripped $ Herewestripboththesymbolicdebuginformationandthe.commentsectionfromtheexecutable file.Wespecifythenameofthestrippedbinaryusingthe-ocommandlineswitch.Youcanseethat the resulting size of the stripped binary is less than half of its original size. Of course, for larger applications,thisspacesavingscanbeevenmoresignificant.ArecentLinuxkernelcompiledwith debuginformationwaslargerthan18MB.AfterstrippingasinListing15-2,theresultingbinarywas slightlylargerthan2MB! Fordebugginginthisfashion,youplacethestrippedversionofthebinaryonyourtargetsystem andkeepalocalunstrippedcopyonyourdevelopmentworkstationcontainingsymbolicinformation neededfordebugging.Youusegdbserveronyourtargetboardtoprovideaninterfacebacktoyour developmenthostwhereyourunthefull-blownversionofGDBonyournonstrippedbinary. 15.2.1.gdbserver UsinggdbserverallowsyoutorunGDBfromadevelopmentworkstationratherthanonthetarget embedded Linux platform. This configuration has obvious benefits. For starters, it is common that your development workstation has far more CPU power, memory, and hard-drive storage than the embeddedplatform.Inaddition,itiscommonforthesourcecodeforyourapplicationunderdebugto existonthedevelopmentworkstationandnotontheembeddedplatform. gdbserver is a small program that runs on the target board and allows remote debugging of a process on the board. It is invoked on the target board specifying the program to be debugged, as well as an IP address and port number on which it will listen for connection requests from GDB. Listing15-3showsthestartupsequenceonthetargetboard. Listing15-3.StartinggdbserveronTargetBoard $gdbserverlocalhost:2001websdemo-stripped Processwebsdemo-strippedcreated;pid=197 Listeningonport2001 ThisparticularexamplestartsgdbserverconfiguredtolistenforanEthernetTCP/IPconnection onport2001,readytodebugourstrippedbinaryprogramcalledwebsdemo-stripped. Fromourdevelopmentworkstation,welaunchGDB,passingitthenameofthebinaryexecutable containingsymbolicdebuginformationthatwewanttodebugasanargument.AfterGDBstartsup, weissueacommandtoconnecttotheremotetargetboard.Listing15-4showsthissequence. Listing15-4.StartingRemoteGDBSession $xscale_be-gdb-qwebsdemo (gdb)targetremote192.168.1.141:2001 Remotedebuggingusing192.168.1.141:2001 0x40000790in??() (gdb)pmain<<<<displayaddressofmainfunction $1={int(int,char**)}0x12b68<main> (gdb)bmain<<<<Placebreakpointatmain() Breakpoint1at0x12b80:filemain.c,line72. (gdb) The sequence in Listing 15-4 invokes cross-gdb on your development host. When GDB is running,weissuethegdbtargetremotecommand.ThiscommandcausesGDBtoinitiateaTCP/IP connectionfromyourdevelopmentworkstationtoyourtargetboard,withtheindicatedIPaddresson port2001.Whengdbserveracceptstheconnectionrequest,itprintsalinesimilartothis: Remotedebuggingfromhost192.168.0.10 NowGDBisconnectedtothetargetboard'sgdbserverprocess,readytoacceptcommandsfrom GDB. The rest of the session is exactly the same as if you were debugging an application locally. This is a powerful tool, allowing you to use the power of your development workstation for the debug session, leaving only a small, relatively unobtrusive GDB stub and your program being debuggedonthetargetboard.Incaseyouwerewondering,gdbserverforthisparticularARMtarget isonly54KB. root@coyote:~#ls-l/usr/bin/gdbserver -rwxr-xr-x1rootroot54344Jul232005/usr/bin/gdbserver Thereisonecaveat,anditisthesubjectofafrequentlyaskedquestion(FAQ)onmanymailing lists.YoumustbeusingaGDBonyourdevelopmenthostthatwasconfiguredasacross-debugger.It is a binary program that runs on your development workstation but understands binary executable imagescompiledforanotherarchitecture.Thisisanimportantandfrequentlyoverlookedfact.You cannot debug a PowerPC target with a native GDB such as that found in a typical Red Hat Linux installation.YoumusthaveaGDBconfiguredforyourhostandtargetcombination. WhenGDBisinvoked, itdisplays abanner consistingofseverallinesofinformation andthen displaysitscompiledconfiguration.Listing15-5isanexampleoftheGDBusedforsomeexamples in this book, which is part of an embedded Linux distribution provided by MontaVista Software configuredforPowerPCcross-development. Listing15-5.Invocationofcross-gdb $ppc_82xx-gdb GNUgdb6.0(MontaVista6.0-8.0.4.03005322003-12-24) Copyright2003FreeSoftwareFoundation,Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions.ThereisabsolutelynowarrantyforGDB.Type"showwarranty"fordetails.ThisGDB wasconfiguredas"--host=i686-pc-linux-gnu--target=powerpc-hardhat-linux". (gdb) Notice the last lines of this GDB startup message. This is the configuration compiled into this versionofGDB.ItwascompiledtoexecuteonaPentium(i686)PChostrunningGNU/Linuxandto debugbinaryprogramscompiledforaPowerPCprocessorrunningGNU/Linux.Thisisspecifiedby the--hostand--targetvariablesdisplayedbythebannertext,andisalsoapartoftheconfiguration stringpassedto./configurewhenbuildingGDB. 15.3.DebuggingwithSharedLibraries Now that you understand how to invoke a remote debug session using GDB on the host and gdbserver on the target, we turn our attention to the complexities of shared libraries and debug symbols. Unless your application is a statically linked executable (linked with the -static linker command line switch), many symbols in your application will reference code outside your application.ObviousexamplesincludetheuseofstandardClibraryroutinessuchasfopen,printf, malloc, and memcpy. Less obvious examples might include calls to application-specific functions, suchasjack_transport_locate()(aroutinefromtheJACKlow-latencyaudioserver),whichcallsa libraryfunctionoutsidethestandardClibraries. Tohavesymbolsfromtheseroutinesavailable,youmustsatisfytworequirementsforGDB: •Youmusthavedebugversionsofthelibrariesavailable. •GDBmustknowwheretofindthem. Ifyoudon'thavedebugversionsofthelibrariesavailable,youcanstilldebugyourapplication; youjustwon'thaveanydebuginformationavailableforlibraryroutinescalledbyyourapplication. Often this is perfectly acceptable, unless, of course, you are developing a shared library object as partofyourembeddedproject. LookbackatListing15-4,whereweinvokedGDBonaremotetarget.AfterGDBconnectedvia thetargetremotecommand,GDBissuedatwo-lineresponse: Remotedebuggingusing192.168.1.141:2001 0x40000790in??() ThisconfirmsthatGDBconnectedtoourtargetattheindicatedIPaddressandport.GDBthen reportsthelocationoftheprogramcounteras0x40000790.Whydowegetquestionmarksinsteadof a symbolic location? Because this is the Linux dynamic loader (ld-x.y.z.so), and on this particular platform,wedonothavedebugsymbolsavailableforthissharedlibrary.Howdoweknowthis? Recallourintroductionofthe/procfilesystemfromChapter9,"FileSystems."Oneofthemore useful entries was the maps entry (see Listing 9-16, in Chapter 9) in the per-process directory structure. We know the process ID (PID) of our target application from the gdbserver output in Listing15-3.OurprocesswasassignedPID197.Giventhat,wecanseethememorysegmentsinuse rightafterprocessstartup,asshowninListing15-6. Listing15-6.InitialTargetMemorySegmentMapping root@coyote:~#cat/proc/197/maps 00008000-00026000r-xp0000000000:0e4852444./websdemo-stripped 0002d000-0002e000rw-p0001d00000:0e4852444./websdemo-stripped 40000000-40017000r-xp0000000000:0a4982583/lib/ld-2.3.3.so 4001e000-40020000rw-p0001600000:0a4982583/lib/ld-2.3.3.so bedf9000-bee0e000rwxpbedf900000:000[stack] root@coyote:~# Here we see the target websdemo-stripped application occupying two memory segments. The firstistheread-onlyexecutablesegmentat0x8000,andthesecondisaread-writedatasegmentat 0x2d000.Thethirdmemorysegmentistheoneofinterest.ItistheLinuxdynamiclinker'sexecutable codesegment.Noticethatitstartsataddress0x40000000.Ifweinvestigatefurther,wecanconfirm thatGDBisactuallysittingatthefirstlineofcodeforthedynamiclinker,beforeanycodefromour ownapplicationhasbeenexecuted.Usingourcrossversionofreadelf,wecanconfirmthestarting addressofthelinkerasfollows: #xscale_be-readelf-Sld-2.3.3.so|grep\.text [9].textPROGBITS00000790000790012c6c00AX0016 Fromthisdata,weconcludethattheaddressGDBreportsonstartupisthefirstinstructionfrom ld-2.3.3.so,theLinuxdynamiclinker/loader.Youcanusethistechniquetogetroughideasofwhere yourcodeisifyoudon'thavesymbolicdebuginformationforaprocessorsharedlibrary. Rememberthatweareexecutingthiscrossreadelfcommandonourdevelopmenthost.Therefore, the ld-2.3.3.so file, itself an XScale binary object, must be accessible to your development host. Most typically, this file resides on your development host, and is a component of your embedded Linuxdistributioninstalledonyourhost. 15.3.1.SharedLibraryEventsinGDB GDB can alert you to shared library events. This can be useful for understanding your application'sbehaviororthebehavioroftheLinuxloader,orforsettingbreakpointsinsharedlibrary routines you want to debug or step through. Listing 15-7 illustrates this technique. Normally, the completepathtothelibraryisdisplayed.Thislistinghasbeeneditedforbetterreadability. Listing15-7.StoppingGDBonSharedLibraryEvents $xscale_be-gdb-qwebsdemo (gdb)targetremote192.168.1.141:2001 Remotedebuggingusing192.168.1.141:2001 0x40000790in??() (gdb)ishared<<<Displayloadedsharedlibs Nosharedlibrariesloadedatthistime. (gdb)bmain<<<Breakatmain Breakpoint1at0x12b80:filemain.c,line72. (gdb)c Continuing. Breakpoint1,main(argc=0x1,argv=0xbec7fdc4)atmain.c:72 72intlocalvar=9; (gdb)ishared FromToSymsReadSharedObjectLibrary 0x400333000x4010260cYes/opt/mvl/.../lib/tls/libc.so.6 0x400007900x400133fcYes/opt/mvl/.../lib/ld-linux.so.3 (gdb)setstop-on-solib-events1 (gdb)c Continuing. Stoppedduetosharedlibraryevent (gdb)ishared FromToSymsReadSharedObjectLibrary 0x400333000x4010260cYes/opt/mvl/.../lib/tls/libc.so.6 0x400007900x400133fcYes/opt/mvl/.../lib/ld-linux.so.3 0x4012bad80x40132104Yes/opt/mvl/.../libnss_files.so.2 (gdb) Whenthedebugsessionisfirststarted,ofcourse,nosharedlibrariesareloaded.Youcanseethis with the first i shared command. This command displays the shared libraries that are currently loaded.Settingabreakpointatourapplication'smain()function,weseethattwosharedlibrariesare nowloaded.ThesearetheLinuxdynamiclinker/loaderandthestandardClibrarycomponentlibc. Fromhere,weissuethesetstop-on-solib-eventcommandandcontinueprogramexecution.When theapplicationtriestoexecuteafunctionfromanothersharedlibrary,thatlibraryisloaded.Incase you are wondering, the gethostbyname() function is encountered and causes the next shared object load. This example illustrates an important cross-development concept. The binary application (ELF image) running on the target contains information on the libraries it needs to resolve its external references. We can view this information easily using the ldd command introduced in Chapter 11, "BusyBox,"anddetailedinChapter13.Listing15-8showstheoutputoflddinvokedfromthetarget board. Listing15-8.lddExecutedonTargetBoard root@coyote:/workspace#lddwebsdemo libc.so.6=>/lib/tls/libc.so.6(0x40020000) /lib/ld-linux.so.3(0x40000000) root@coyote:/workspace# Noticethatthepathstothesharedlibrariesonthetargetareabsolutepathsstartingat/libonthe rootfilesystem.ButGDBrunningonyourhostdevelopmentworkstationcannotusethesepathsto findthelibraries.YoushouldrealizethattodosowouldresultinyourhostGDBloadinglibraries from the wrong architecture. Your host is likely x86, whereas, in this example, the target is ARM XScale. Ifyouinvokeyourcrossversionofldd,youwillseethepathsthatwerepreconfiguredintoyour toolchain.Yourtoolchainmusthaveknowledgeofwherethesefilesexistonyourhostdevelopment system.[101]Listing15-9illustratesthis.Again,wehaveeditedthelistingforreadability;longpaths havebeenabbreviated. Listing15-9.lddExecutedonDevelopmentHost $xscale_be-lddwebsdemo libc.so.6=>/opt/mvl/.../xscale_be/target/lib/libc.so.6(0xdead1000) ld-linux.so.3=>/opt/mvl/.../xscale_be/target/lib/ld-linux.so.3(0xdead2000) $ Your cross toolchain should be preconfigured with these library locations. Not only does your hostGDBneedtoknowwheretheyarelocated,but,ofcourse,yourcompilerandlinkeralsoneed thisknowledge.[102] GDB can tell you where it is configured to look for these libraries using the showsolib-absolute-prefixcommand: (gdb)showsolib-absolute-prefix Prefixforloadingabsolutesharedlibrarysymbolfilesis "/opt/mvl/pro/devkit/arm/xscale_be/target". (gdb) You can set or change where GDB searches for shared libraries using the GDB commands set solib-absolute-prefix and set solib-search-path. If you are developing your own shared library modulesorhavecustomlibrarylocationsonyoursystem,youcanusesolib-search-pathtoinstruct GDB where to look for your libraries. For more details about these and other GDB commands, consulttheonlineGDBmanualreferencedattheendofthischapterinSection15.6.1,"Suggestions forAdditionalReading." One final note about ldd. You might have noticed the addresses from Listing 15-8 and 15-9 associated with the libraries. ldd displays the load address for the start of these code segments as theywouldbeiftheprogramwereloadedbytheLinuxdynamiclinker/loader.Executedonthetarget, the addresses in Listing 15-5 make perfect sense, and we can correlate these with the /proc/<pid>/maps listing of the running process on the target. Listing 15-10 displays the memory segmentsforthistargetprocessafteritiscompletelyloadedandrunning. Listing15-10.MemorySegmentsfrom/proc/<pid>/mapsonTarget root@coyote:~#cat/proc/197/maps 00008000-00026000r-xp0000000000:0e4852444/workspace/websdemo-stripped 0002d000-0002e000rw-p0001d00000:0e4852444/workspace/websdemo-stripped 0002e000-0005e000rwxp0002e00000:000[heap] 40000000-40017000r-xp0000000000:0a4982583/lib/ld-2.3.3.so 40017000-40019000rw-p4001700000:000 4001e000-4001f000r--p0001600000:0a4982583/lib/ld-2.3.3.so 4001f000-40020000rw-p0001700000:0a4982583/lib/ld-2.3.3.so 40020000-4011d000r-xp0000000000:0a4982651/lib/tls/libc-2.3.3.so 4011d000-40120000---p000fd00000:0a4982651/lib/tls/libc-2.3.3.so 40120000-40124000rw-p000f800000:0a4982651/lib/tls/libc-2.3.3.so 40124000-40126000r--p000fc00000:0a4982651/lib/tls/libc-2.3.3.so 40126000-40128000rw-p000fe00000:0a4982651/lib/tls/libc-2.3.3.so 40128000-4012a000rw-p4012800000:000 4012a000-40133000r-xp0000000000:0a4982652/lib/tls/libnss_files-2.3.3.so 40133000-4013a000---p0000900000:0a4982652/lib/tls/libnss_files-2.3.3.so 4013a000-4013b000r--p0000800000:0a4982652/lib/tls/libnss_files-2.3.3.so 4013b000-4013c000rw-p0000900000:0a4982652/lib/tls/libnss_files-2.3.3.so becaa000-becbf000rwxpbecaa00000:000[stack] root@coyote:~# Notice the correlation of the target ldd output from Listing 15-8 to the memory segments displayedinthe/procfilesystemforthisprocess.Thestart(beginningof.textsegment)oftheLinux loaderis0x40000000andthestartoflibcisat0x40020000.Thesearethevirtualaddresseswhere theseportionsoftheapplicationhavebeenloaded,andarereportedbythetargetinvocationofldd. However,theloadaddressesreportedbythecrossversionoflddinListing15-9(0xdead1000and 0xdead2000)aretheretoremindyouthattheselibrariescannotbeloadedonyourhostsystem(they areARMarchitecturebinaries),andtheloadaddressesaresimplyplaceholders. 15.4.DebuggingMultipleTasks Generallythedeveloperispresentedwithtwodifferentdebuggingscenarioswhendealingwith multiplethreadsofexecution.Processescanexistintheirownaddressspaceorcanshareanaddress space (and other system resources) with other threads of execution. The former (independent processesnotsharingcommonaddressspace)mustbedebuggedusingseparateindependentdebug sessions. Nothing prevents you from using gdbserver on multiple processes on your target system, andusingaseparateinvocationofGDBonyourdevelopmenthosttocoordinateadebugsessionfor multiplecooperatingbutindependentprocesses. 15.4.1.DebuggingMultipleProcesses When a process being debugged under GDB uses the fork() system call[103] to spawn a new process,GDBcantaketwocoursesofaction.Itcancontinuetocontrolanddebugtheparentprocess, oritcanstop debugging theparentprocessandattachtothenewlyformedchildprocess.Youcan controlthisbehaviorusingthesetfollow-fork-modecommand.Thetwomodesarefollowparentand followchild.ThedefaultbehaviorisforGDBtofollowtheparent.Inthiscase,thechildprocess executesimmediatelyuponasuccessfulfork. Listing 15-11 reproduces a snippet of a simple program that forks multiple processes from its main()routine. Listing15-11.Usingfork()toSpawnaChildProcess ... for(i=0;i<MAX_PROCESSES;i++){ /*Creatingchildprocess*/ pid[i]=fork();/*Parentgetsnon-zeroPID*/ if(pid[i]==-1){ perror("forkfailed"); exit(1); } if(pid[i]==0){/*Indicateschild'scodepath*/ worker_process();/*Theforkedprocesscallsthis*/ } } /*Parent'smaincontrolloop*/ while(1){ ... } This simple loop creates MAX_THREADS new processes using the fork() system call. Each newlyspawnedprocessexecutesabodyofcodedefinedbythefunctionworker_process().Whenrun underGDBinthedefaultmode,GDBdetectsthecreationofthenewthreadsofexecution(processes) butremainsattachedtotheparent'sthreadofexecution.Listing15-12illustratesthisGDBsession. Listing15-12.GDBinfollow-fork-mode=parent (gdb)targetremote192.168.1.141:2001 0x40000790in??() (gdb)bmain Breakpoint1at0x8888:fileforker.c,line104. (gdb)c Continuing. [NewThread356] [SwitchingtoThread356] Breakpoint1,main(argc=0x1,argv=0xbe807dd4)atforker.c:104 104time(&start_time); (gdb)bworker_process Breakpoint2at0x8784:fileforker.c,line45. (gdb)c Continuing. Detachingafterforkfromchildprocess357. Detachingafterforkfromchildprocess358. Detachingafterforkfromchildprocess359. Detachingafterforkfromchildprocess360. Detachingafterforkfromchildprocess361. Detachingafterforkfromchildprocess362. Detachingafterforkfromchildprocess363. Detachingafterforkfromchildprocess364. Noticethat eightchild processeswerespawned,withPIDvaluesfrom357 to364.Theparent process was instantiated with PID 356. When the breakpoint in main() was hit, we entered a breakpointattheworker_process()routine,whicheachchildprocessexecutesuponfork().Letting the program continue from main, we see each of the new processes spawned and detached by the debugger.TheyneverhitthebreakpointbecauseGDBisattachedtothemainprocess,whichnever executestheworker_process()routine. If you need to debug each process, you must execute a separate independent GDB session and attachtothechildprocessafteritisforked().TheGDBdocumentationreferencedattheendofthis chapteroutlinesausefultechniquetoplaceacalltosleep()inthechildprocess,givingyoutimeto attach a debugger to the new process. Attaching to a new process is explained in Section 15.5.2, "AttachingtoaRunningProcess." Ifyousimplyneedtofollowthechildprocess,setthefollow-fork-modetofollowchildbefore yourparentreachesthefork()systemcall.Listing15-13showsthis. Listing15-13.GDBinfollow-fork-mode=child (gdb)targetremote192.168.1.141:2001 0x40000790in??() (gdb)setfollow-fork-modechild (gdb)bworker_process Breakpoint1at0x8784:fileforker.c,line45. (gdb)c Continuing. [NewThread401] Attachingafterforktochildprocess402. [NewThread402] [SwitchingtoThread402] Breakpoint1,worker_process()atforker.c:45 45intmy_pid=getpid(); (gdb)c Continuing. HereweseetheparentprocessbeinginstantiatedasPID401.Whenthefirstchildisspawnedby thefork()systemcall,GDBdetachessilentlyfromtheparentthreadofexecutionandattachestothe newlyspawnedchildprocesshavingPID402.GDBisnowincontrolofthefirstchildprocessand honors the breakpoint set at worker_process(). Notice, however, that the other child processes spawnedbythecodesnippetfromListing15-11arenotdebuggedandcontinuetoruntotheirown completion. Insummary,usingGDBinthisfashion,youarelimitedtodebuggingasingleprocessatatime. You can debug through the fork() system call, but you have to decide which thread of execution to followthroughthefork()call,eithertheparentorthechild.Asmentionedintheintroductiontothis section,youcanusemultipleindependentGDBsessionsifyoumustdebugmorethanonecooperating processatatime. 15.4.2.DebuggingMultithreadedApplications IfyourapplicationusesthePOSIXthreadlibraryforitsthreadingfunctions,GDBhasadditional capabilitiestohandleconcurrentdebuggingofamultithreadedapplication.TheNativePosixThread Library(NPTL)hasbecomethedefactostandardthreadlibraryinuseonLinuxsystems,including embeddedLinuxsystems.Therestofthisdiscussionassumesthatyouareusingthisthreadlibrary. For this section, we use a demonstration program that spawns a number of threads using the pthread_create()libraryfunctioninasimpleloop.Afterthethreadsarespawned,themain()routine simplywaitsforkeyboardinputtoterminatetheapplication.Eachthreaddisplaysashortmessageon the screen and sleeps for a predetermined time. Listing 15-14 shows the startup sequence on the targetboard. Listing15-14.TargetThreadsDemoStartup root@coyote:/workspace#gdbserverlocalhost:2001./tdemo Process./tdemocreated;pid=671 Listeningonport2001 Remotedebuggingfromhost192.168.1.10 ^^^^^Previousthreelinesdisplayedbygdbserver tdemomain()entered:Mypidis671 Startingworkerthread0 Startingworkerthread1 Startingworkerthread2 Startingworkerthread3 As in our previous examples, gdbserver prepares the application for running and waits for a connection from our host-based cross-gdb. When GDB connects, gdbserver reports the connection with the Remote debugging... message. Now we start GDB on the host and connect. Listing 15-15 reproducesthishalfofthesession. Listing15-15.HostGDBConnectingtoTargetThreadsDemo $xscale_be-gdb-qtdemo (gdb)targetremote192.168.1.141:2001 0x40000790in??() (gdb)btdemo.c:97 Breakpoint1at0x88ec:filetdemo.c,line97. (gdb)c Continuing. [NewThread1059] [NewThread1060] [NewThread1061] [NewThread1062] [NewThread1063] [SwitchingtoThread1059] Breakpoint1,main(argc=0x1,argv=0xbefffdd4)attdemo.c:98 98intc=getchar(); (gdb) Hereweconnecttothetarget(resultinginthe"Remotedebugging..."messageinListing15-14), setabreakpointjustpasttheloopwherewespawnedthenewthreads,andcontinue.Whenthenew thread is created, GDB displays a notice along with the thread ID. Thread 1059 is the TDemo application,doingitsworkdirectlyfromthemain()function.Threads1060through1063arethenew threadscreatedfromthecalltopthread_create(). WhenGDBhitsthebreakpoint,it displaysthemessage [SwitchingtoThread1059],indicating that this was the thread of execution that encountered the breakpoint. It is the active thread for the debuggingsession,referredtoasthecurrentthreadintheGDBdocumentation. GDBenablesustoswitchbetweenthreadsandperformtheusualdebuggingoperationssuchas setting additional breakpoints, examining data, displaying a backtrace, and working with the individual stack frames within the current thread. Listing 15-16 provides examples of these operations,continuingdirectlywithourdebuggingsessionstartedinListing15-15. Listing15-16.GDBOperationsonThreads ... (gdb)c Continuing. <<<Ctl-Ctointerruptprogramexecution ProgramreceivedsignalSIGINT,Interrupt. 0x400db9c0inread()from/opt/mvl/.../lib/tls/libc.so.6 (gdb)ithreads 5Thread10630x400bc714innanosleep() from/opt/mvl/.../lib/tls/libc.so.6 4Thread10620x400bc714innanosleep() from/opt/mvl/.../lib/tls/libc.so.6 3Thread10610x400bc714innanosleep() from/opt/mvl/.../lib/tls/libc.so.6 2Thread10600x400bc714innanosleep() from/opt/mvl/.../lib/tls/libc.so.6 *1Thread10590x400db9c0inread() from/opt/mvl/.../lib/tls/libc.so.6 (gdb)thread4<<<MakeThread4thecurrentthread [Switchingtothread4(Thread1062)] #00x400bc714innanosleep() from/opt/mvl/.../lib/tls/libc.so.6 (gdb)bt #00x400bc714innanosleep() from/opt/mvl/.../lib/tls/libc.so.6 #10x400bc4a4in__sleep(seconds=0x0)atsleep.c:137 #20x00008678ingo_to_sleep(duration=0x5)attdemo.c:18 #30x00008710inworker_2_job(random=0x5)attdemo.c:36 #40x00008814inworker_thread(threadargs=0x2)attdemo.c:67 #50x40025244instart_thread(arg=0xfffffdfc)atpthread_create.c:261 #60x400e8fa0inclone()at../sysdeps/unix/sysv/linux/arm/clone.S:82 #70x400e8fa0inclone()at../sysdeps/unix/sysv/linux/arm/clone.S:82 (gdb)frame3 #30x00008710inworker_2_job(random=0x5)attdemo.c:36 36go_to_sleep(random); (gdb)l<<<Generatelistingofwhereweare 31} 32 33staticvoidworker_2_job(intrandom) 34{ 35printf("t2sleepingfor%d\n",random); 36go_to_sleep(random); 37} 38 39staticvoidworker_3_job(intrandom) 40{ (gdb) Afewpointsareworthmentioning.GDBassignsitsownintegervaluetoeachthreadanduses these values to reference the individual threads. When a breakpoint is hit in a thread, all threads withintheprocessarehaltedforexamination.GDBmarksthecurrentthreadwithanasterisk(*).You cansetuniquebreakpointswithineachthreadassuming,ofcourse,thattheyexistinauniquecontext. Ifyousetabreakpointinacommonportionofcodewhereallthreadsexecute,thethreadthathitsthe breakpointfirstisarbitrary. The GDB user documentation referenced at the end of this chapter contains more useful informationrelatedtodebugginginamultithreadedenvironment. 15.4.3.DebuggingBootloader/FlashCode DebuggingFlashresidentcodepresentsitsownuniquechallenges.Themostobviouslimitation isthewayinwhichGDBandgdbservercooperateinsettingtargetbreakpoints.Whenwediscussed the GDB remote serial protocol in Chapter 14, you learned how breakpoints are inserted into an application.[104] GDB replaces the opcode at the breakpoint location with an architecture-specific opcodethatpassescontroltothedebugger.However,inROMorFlash,GDBcannotoverwritethe opcode,sothismethodofsettingbreakpointsisuseless. Mostmodernprocessorscontainsomenumberofdebugregistersthatcanbeusedtogetaround this limitation. These capabilities must be supported by architecture-and processor-specific hardwareprobesorstubs.ThemostcommontechniquefordebuggingFlashandROMresidentcode is to use JTAG hardware probes. These probes support the setting of processor-specific hardware breakpoints. This topic was covered in detail in Chapter 14. Refer back to Section 14.4.2, "DebuggingwithaJTAGProbe,"fordetails. 15.5.AdditionalRemoteDebugOptions Sometimesyoumightwanttouseaserialportforremotedebugging.Forothertasks,youmight find it useful to attach the debugger to a process that is already running. These simple but useful operationsaredetailedhere.[105] 15.5.1.DebuggingviaSerialPort Debugging via serial port is quite straightforward. Of course, you must have a serial port availableonyourtargetthatisnotbeingusedbyanotherprocess,suchasaserialconsole.Thesame limitationappliestoyourhost.Aserialportmustbeavailable.Ifbothoftheseconditionscanbemet, simplyreplacetheIP:Portspecificationpassedtogdbserverwithaserialportspecification.Usethe sametechniquewhenconnectingtoyourtargetfromyourhost-basedGDB. Onyourtarget: root@coyote:/workspace#gdbserver/dev/ttyS0./tdemo Process./tdemocreated;pid=698 Remotedebuggingusing/dev/ttyS0 Fromyourhost: $xscale_be-gdb-qtdemo (gdb)targetremote/dev/ttyS1 Remotedebuggingusing/dev/ttyS1 0x40000790in??() 15.5.2.AttachingtoaRunningProcess Itisoftenadvantageoustoconnecttoaprocesstoexamineitsstatewhileitisrunninginsteadof killingtheprocessandstartingitagain.Withgdbserver,itistrivial: root@coyote:/workspace#psax|greptdemo 1030pts/0Sl+0:00./tdemo root@coyote:/workspace#gdbserverlocalhost:2001--attach1030 Attached;pid=1030 Listeningonport2001 When you are finished examining the process under debug, you can issue the gdb detach command. This detaches the gdbserver from the application on the target and terminates the debug session.Theapplicationcontinueswhereitleftoff.Thisisaveryusefultechniqueforexamininga running program. Be aware, though, that when you attach to the process, it halts, waiting for instructionsfromyou.Itwillnotresumeexecutionuntilinstructedtodoso,usingeitherthecontinue commandorthedetachcommand.Alsonotethatyoucanusethedetachcommandatalmostanytime toendthedebugsessionandleavetheapplicationrunningonthetarget. 15.6.ChapterSummary • Remote (cross) debugging enables symbolic debugging using host development workstation resourcesfortheheavylifting,preservingoftenscarcetargetresources. • gdbserver runs on the target system and acts as the glue between the cross-gdb running on a developmenthostandtheprocessbeingdebuggedonthetarget. •GDBonthehosttypicallyusesIPconnectionsviaEthernettosendandreceivecommandsto gdbserver running on the target. The GDB remote serial protocol is used between GDB and gdbserver. •GDBcanhaltonsharedlibraryeventsandcanautomaticallyloadsharedlibrarysymbolswhen available. Your toolchain should be configured for the default paths on your cross-development system.Alternatively,youcanuseGDBcommandstosetthesearchpathsforsharedlibraryobjects. • GDB can be used to debug multiple independent processes via multiple concurrent GDB sessions. •GDBcanbeconfiguredtofollowaforkedprocessonafork()systemcall.Itsdefaultmodeisto continuetodebugtheparentthatis,thecalleroffork(). • GDB has features to facilitate debugging multithreaded applications written to POSIX thread APIs.ThecurrentdefaultLinuxthreadlibraryistheNativePosixThreadsLibrary(NPTL). •GDBsupportsattachingtoanddetachingfromanalreadyrunningprocess. 15.6.1.SuggestionsforAdditionalReading GDB:TheGNUProjectDebugger OnlineDocumentation http://sourceware.org/gdb/onlinedocs/ GDBPocketReference ArnoldRobbins O'ReillyMedia,2005 Chapter16.PortingLinux ItisnotdifficulttoportLinuxtoanewhardwareplatform.TheLinuxsourcetreecontainsports for numerous boards spanning more than 20 architectures and many more individual processors. Knowingwheretostartisoftenthehardestpart. This chapter covers the basics of porting Linux to a custom board providing support for basic Ethernetandserialconsoleoperation.WeexaminetheorganizationoftheLinuxsourcecodefroman architectural and platform perspective. We then delve into the early kernel initialization code to understandthemechanismsprovidedforplatforminitialization.Finally,welookatatypicalporting efforttoacustomPowerPChardwareplatform. 16.1.LinuxSourceOrganization Nottoolongago,therewerenumeroushomes[106]forthevariousversionsofLinux.Therewasa dedicatedplaceforthePowerPCversionofLinux,onefortheARMversion,andsoon.Thiswasn't necessarilybydesign,butbynecessity.Ittooktimetomergethevariousarchitectureinfrastructure andfeaturesintothemainlinekernel,andhavingaseparatesourcetreemeantquickeraccesstothe latestfeaturesinagivenarchitecture. ThekerneldevelopershavegonetogreatlengthstounifytheLinuxkernelsourcecodetobring togetherthedisparatearchitecturesunderonecommonsourcetree.Withfewexceptions,thisisthe casetodaywiththeLinux2.6source.Itispossibletodownloadandcompileworkingkernelsfora varietyofprocessorsandindustry-standardreferenceboardsdirectlyfromwww.kernel.org. 16.1.1.TheArchitectureBranch InChapter4,"TheLinuxKernel:ADifferentPerspective,"weintroducedtheoverallstructureof the Linux kernel source tree. We spend the majority of this chapter examining the architecturespecificbranchoftheLinuxkernelsources.Listing16-1showsthecontentsof.../archfromarecent kernel snapshot. As we pointed out in Chapter 4, the .../arch subdirectory is the second largest in termsofsize,andinarecentLinuxrelease,thelargestintermsoffilecount(excludingthe.../include directory).Onlythe.../driverssubdirectoryislargerinsize. Listing16-1.LinuxKernel.../archDirectoryListing [chris@plutolinux]$ls./arch alphacrisi386m68kpariscs390sparcv850 armfrvia64m68knommuppcshsparc64x86_64 arm26h8300m32rmipsppc64sh64umxtensa Fromthislisting,youcanseesupportfor24separatearchitectureswithintheLinuxkernel.We refertoeachasanarchitecturebranchtofacilitateourdiscussions. Each architecture branch has some common components. For example, each top-level architecturebranchcontainsaKconfigfile.YouwillrecallfromChapter4thatKconfigdrivesthe kernel configuration utility. Of course, each top-level architecture branch also has a corresponding makefile. All the top-level architectures contain a kernel subdirectory because a number of kernel features are architecture dependent. All but two contain an mm subdirectory. This is where the architecture-dependentmemorymanagementcodeisfound. Manytop-levelarchitecturebranchescontainabootsubdirectory,whichisusedtobuild(through its own makefile) a specific bootable target for that architecture. Many also contain mach-* subdirectories. These are used to hold code for specific machines or hardware platforms. Another subdirectorythatappearsfrequentlyinthearchitecturebranchisconfigs.Thissubdirectoryexistsfor many of the more popular architectures and contains default configurations for each supported hardwareplatform. Throughout the rest of this chapter, we focus our discussion and examples on the PowerPC architecture.Itisoneofthemostpopular,withsupportformanyprocessorsandboards.Listing16-2 shows the contents of the configs directory for the .../arch/ppc PowerPC branch of a recent Linux kernelrelease. Listing16-2.PowerPCconfigsDirectoryContents [chris@plutolinux]$ls./arch/ppc/configs/ ads8272_defconfigIVMS8_defconfigprpmc750_defconfig apus_defconfigkatana_defconfigprpmc800_defconfig bamboo_defconfiglite5200_defconfigradstone_defconfig bseip_defconfiglopec_defconfigredwood5_defconfig bubinga_defconfigluan_defconfigredwood6_defconfig chestnut_defconfigmbx_defconfigrpx8260_defconfig common_defconfigmpc834x_sys_defconfigrpxcllf_defconfig cpci405_defconfigmpc8540_ads_defconfigrpxlite_defconfig cpci690_defconfigmpc8548_cds_defconfigsandpoint_defconfig ebony_defconfigmpc8555_cds_defconfigspruce_defconfig ep405_defconfigmpc8560_ads_defconfigstx_gp3_defconfig est8260_defconfigmpc86x_ads_defconfigsycamore_defconfig ev64260_defconfigmpc885ads_defconfigTQM823L_defconfig ev64360_defconfigmvme5100_defconfigTQM8260_defconfig FADS_defconfigocotea_defconfigTQM850L_defconfig gemini_defconfigpmac_defconfigTQM860L_defconfig hdpu_defconfigpower3_defconfigwalnut_defconfig ibmchrp_defconfigpplus_defconfig EachoneoftheseentriesintheconfigsdirectoryofthePowerPCarchitecturebranchrepresentsa specificporttoahardwareplatform.Forexample,walnut_defconfigdefinesthedefaultconfiguration fortheAMCCWalnutPPC405evaluationplatform.Thempc8540_ads_defconfigfilerepresentsthe defaultconfigurationfortheFreescaleMPC8540ADSevaluationboard.AsdescribedinChapter4, tobuildakernelforthesereferenceplatformsyoufirstconfigureyourkernelsourcetreewiththese configurationdefaults,asfollows: $makeARCH=ppcCROSS_COMPILE=ppc_85xx-mpc8540_ads_defconfig This invocation of make (from the top-level Linux directory) configures the kernel source tree withadefaultconfigurationfortheFreescaleMPC8540ADSevaluationboard. OneaspectoftheLinuxkernelsourcetreethathasnotachievedsignificantunificationistheway in which each architecture handles platform-specific files. In the PowerPC branch, you find a platformsdirectorythatcontainsplatform-specificcode.Lookingthroughthisdirectory,youwillsee manysourcefilesnamedaftertherespectivehardwareplatform.Therearealsoafewsubdirectories under.../arch/ppc/platformsforspecificPowerPCvariants. Incontrast,theARMbranchcontainsaseriesofmach-*directories,eachrepresentingaspecific hardwareplatform,whiletheMIPSbranchhasasetofsubdirectoriesnamedforaspecificplatform. 16.2.CustomLinuxforYourBoard WhenweportedU-BoottoanewhardwareplatforminChapter7,"Bootloaders,"wefoundthe configurationthatmostcloselymatchedournewboardandborrowedfromthatport.Weuseasimilar techniquetoportLinuxtoournewboard.WeassumethatthechosenCPUisalreadysupportedinthe kernel.PortingtoanewCPUissignificantlymorechallengingandbeyondthescopeofthisbook. WehavechosentoportLinuxtoacustomcontrollerboardbasedontheFreescaleMPC520032bit embedded PowerPC processor. Looking through the default configurations from a recent Linux release (as depicted in Listing 16-2), we find one that contains the MPC5200 CPU. Because it appearsthatthisistheonlyconfigurationthatsupportsthisprocessor,weuseitasourbaseline. ThehardwareplatformthatweuseforthisexercisewassuppliedcourtesyofUnitedElectronic Industries.TheboardiscalledthePowerDNAController.Ithasasimpleblockdiagram,containing onboardFlashmemory,dynamicRAM,aserialport,andavarietyofI/Odevices,mostlyintegrated intotheMPC5200processor.Figure16-1istheblockdiagramofthePowerDNAController. Figure16-1.UEIPowerDNAControllerboard 16.2.1.PrerequisitesandAssumptions The Linux kernel makes some fundamental assumptions when it is passed control from a bootloader. Most important among them is that the bootloader must have initialized the DRAM controller. Linux does not participate in chip-level SDRAM controller setup. Linux assumes that systemRAMispresentandfullyfunctional.ThePowerDNAControllerwearetargetingcontainsthe U-Bootbootloader,whichhasinitializedtheCPU,DRAM,andotherrelatedhardwarerequiredfor minimalsystemoperation. Thebootloadershouldalsoinitializethesystemmemorymap.Thisisusuallydoneviaasetof processor registers that define what chip select signals are active within a given memory address range.Chapter3intheFreescaleMPC5200User'sGuidedescribestheregistersusedforthistask. Thebootloadermighthaveadditionalhardware-relatedinitializationtasks.Onsomeboards,the kernelassumesthattheserialportisconfigured.Thismakesitpossibletodisplayearlykernelboot messages to the serial port, long before the kernel's own serial driver has been installed. Some architectures and hardware platforms contain functions such as *_serial_putc(), which can send stringstoaserialportthathasbeenpreconfiguredbythebootloaderorbysomesimpleearlykernel setup code. You can find examples of this in the PowerPC architecture branch using grep and searchingforCONFIG_SERIAL_TEXT_DEBUG. Insummary,thefundamentalprerequisiteforportingLinuxtoournewboardisthatabootloader hasbeenportedandinstalledonourboard,andanyboard-specificlow-levelhardwareinitialization hasbeencompleted.ItisnotnecessarytoinitializedevicesforwhichLinuxhasdirectdevicedriver support,suchasEthernetcontrollersorI2Ccontrollers;thekernelhandlesthese. ItisagoodideatoconfigureandbuildyourLinuxkernelfortheboardclosesttoyourown.This providesyouwithaknowngoodstartingpointaLinuxkernelsourcetreeconfiguredforyourboard thatcompileswithouterror.RecallfromChapter5,"KernelInitialization,"thecommandtocompilea Linux2.6kernel: $makeARCH=ppcCROSS_COMPILE=ppc_82xx-uImage ThiscommandlineresultsinaLinuxbootableimagecompatiblewiththeU-Bootbootloader.The uImagetargetspecifiesthis. 16.2.2.CustomizingKernelInitialization Now that we have a baseline kernel source tree from which to start, let's determine where to begin customizing for our particular board. We discovered that for the PowerPC architecture, the board-specificfilesresideinadirectorycalled.../arch/ppc/platforms.Ofcourse,thisisnotstrictly necessary,butifyoueverintendtosubmityourpatchestotheLinuxkerneldevelopmentcommunity forconsideration,properformandconsistencymatter! Wefindintheplatformsdirectoryafilecalledlite5200.c.It'safairlysimplefile,containingtwo datastructuresandfivefunctions.Listing16-3presentsthefunctionsfromthisfile. Listing16-3.Functionsfrom5200PlatformFile lite5200_show_cpuinfo()/*Printsuserspecifiedtextstring*/ lite5200_map_irq()/*Setsh/wspecificINTlogicrouting*/ lite5200_setup_cpu()/*CPUspecificinitialization*/ lite5200_setup_arch()/*Arch.specificinitialization*/ platform_init()/*Machineorboardspecificinit*/ Let'slookathowthesefunctionsareused.Webrieflyexaminedthelow-levelkernelinitialization in Chapter 5. Here we look at the details for a particular architecture. Details differ between architectures,butwhenyoucannavigateone,theotherswillbeeasiertolearn. FromChapter5,wesawtheearlyflowofcontrolonpower-up.Thebootloaderpassedcontrolto thekernel'sbootstraploader,which thenpassedcontroltotheLinuxkernelviathekernel'shead.o module.Heretheplatform-specificinitializationbegins.Listing16-4reproducesthepertinentlines from.../arch/ppc/kernel/head.S. Listing16-4.CallingEarlyMachineInitialization ... /* *Doearlybootinfoparsing,platform-specificinitialization, *andsetuptheMMU. */ mrr3,r31 mrr4,r30 mrr5,r29 mrr6,r28 mrr7,r27 blmachine_init blMMU_init ... Here you can see the assembly language call to machine_init. Of particular significance is the setup of the registers r3 through r7. These registers are expected to contain well-known values, which you will see momentarily. They were stored away very early in the boot sequence to the PowerPC general-purpose registers r27 through r31. Here they are reloaded from these stored values. Themachine_init()functionisdefinedinaCfilecalledsetup.c,inthesamearchitecture-specific kerneldirectory:.../arch/ppc/kernel/setup.c.ThestartofthisroutineisreproducedhereinListing165. Listing16-5.Functionmachine_init()insetup.c void__initmachine_init(unsignedlongr3,unsignedlongr4,unsignedlongr5,unsignedlongr6, unsignedlongr7){ #ifdefCONFIG_CMDLINE strlcpy(cmd_line,CONFIG_CMDLINE,sizeof(cmd_line)); #endif/*CONFIG_CMDLINE*/ #ifdefCONFIG_6xx ppc_md.power_save=ppc6xx_idle; #endif #ifdefCONFIG_POWER4 ppc_md.power_save=power4_idle; #endif platform_init(r3,r4,r5,r6,r7); if(ppc_md.progress) ppc_md.progress("idmach():done",0x200); } Thereissomeveryusefulknowledgeinthissimplefunction.First,noticethattheparametersto machine_init()representthePowerPCgeneral-purposeregistersr3throughr7.[107]Yousawthatthey wereinitializedjustbeforethemachinelanguagecalltomachine_init.AsyoucanseefromListing 16-5,theseregistervaluesarepassedunmodifiedtoplatform_init().Weneedtomodifythisfunction forourplatform.(Wehavemoretosayaboutthatinamoment.) Listing16-5alsocontainssomemachine-specificcallsforpower-managementfunctions.Ifyour kernelisconfiguredforPowerPC6xxsupport(CONFIG_6xxdefinedinyour.configfile),apointer toamachine-specificpower-managementfunction(ppc6xx_idle)isstoredinastructure.Similarly,if your kernel is configured for a PowerPC G5 core (CONFIG_POWER4), a pointer to its machinespecific power-management routine is stored in the same structure member. This structure is describedinSection16.3.3,"Machine-DependentCalls." 16.2.3.StaticKernelCommandLine Oneofthemoreinterestingoperationsinthemachine_init()functionreproducedinListing16-5 is to store the default kernel command line. This operation is enabled if CONFIG_CMDLINE is enabledinyourkernelconfiguration.Onsomeplatforms,thebootloaderdoesnotsupplythekernel command line. In these cases, the kernel command line can be statically compiled into the kernel. Figure16-2illustratestheconfigurationoptionsforthis. Figure16-2.Defaultkernelcommandline Enable "Default bootloader kernel arguments" in the configuration in Figure 16-2 and edit the "Initialkernelcommandstring"asshown.Thisresultsinasetofentriesinthe.configfile,asshown inListing16-6. Listing16-6.ConfigurationforDefaultKernelCommandLine ... CONFIG_CMDLINE_BOOL=y CONFIG_CMDLINE="console=ttyS0root=/dev/ram0rw" ... TheellipsesinListing16-6indicatethatwehavetakenonlyasmallsnippetofthe.configfile. Whentheseconfigurationsymbolsareprocessedbythekernelbuildsystem,theybecomeentriesin the.../include/linux/autoconf.hfile,asdetailedinListing16-7. Listing16-7.Fileautoconf.hEntriesforDefaultKernelCommandLine ... #defineCONFIG_CMDLINE_BOOL1 #defineCONFIG_CMDLINE"console=ttyS0root=/dev/ram0rw" ... NowreferringbacktoListing16-5,wehavethefollowingline: strlcpy(cmd_line,CONFIG_CMDLINE,sizeof(cmd_line)); You can see that this kernel-based string-copy function copies the string defined by CONFIG_CMDLINEintoaglobalkernelvariablecalledcmd_line.Thisisimportantbecausemany functions and device drivers might need to examine the kernel command line early in the boot sequence.Theglobalvariablecmd_lineishiddenawayatthestartofthe.datasection,definedinthe assemblerfile.../arch/ppc/kernel/head.S. A subtle detail is worth mentioning here. Looking back at Listing 16-4, we see that the machine_initassemblylanguagecallismadebeforethecalltoMMU_init.Thatmeansthatanycode we are able to run from machine_init is executed in a context with limited support for accessing memory.Manyoftoday'sprocessorsthatcontainanMMUcannotaccessanymemorywithoutsome initialmappingviahardwareregistersintheprocessor.[108]Typically,asmallamountofmemoryis made available at boot time to accommodate loading and decompressing the kernel and a ramdisk image. Trying to access code or data beyond these early limits will fail. Each architecture and platformmighthavedifferentearlylimitsforaccessingmemory.Valuesontheorderof8to16MB are not untypical. We must remember that any code we execute from machine_init, including our platforminitialization,takesplaceinthiscontext.Ifyouencounterdataaccesserrors(PowerPCDSI exception[109])whiledebuggingyournewkernelport,youshouldimmediatelysuspectthatyouhave notproperlymappedthememoryregionyourcodeistryingtoaccess. 16.3.PlatformInitialization Followingisaquickreviewofthecodeflowduringearlyinitialization.Figure16-3showsthe flowofexecutionfromthebootloaderorbootstraploadertoyourplatform-initializationcode. Figure16-3.Platforminitializationflowofcontrol Thefileshead.Sandsetup.carebothfoundinthe.../arch/ppc/kerneldirectoryforthePowerPC architecture. Our custom platform-initialization file will be placed in the .../arch/ppc/platforms directory.InFigure16-3,itisrepresentedbythefilemyplat.c.Wearenowinapositiontoexamine theplatform-specificinitializationfileindetail. In Listing 16-3, we listed the functions in the lite5200.c platform-initialization file. Every functionexceptplatform_init()isdeclaredasstatic.Therefore,asshowninFigure16-3,thisisthe entrypointfortheplatform-initializationfile.Therestofthefunctionsinthefilearereferencedonly fromwithinthefileitself. Let's examine the entry function platform_init(). Listing 16-8 reproduces the platform_init() functionfromthelite5200.cfile. Listing16-8.Lite5200platform_initFunction void__initplatform_init(unsignedlongr3,unsignedlongr4,unsignedlongr5,unsignedlongr6, unsignedlongr7){ /*GenericMPC52xxplatforminitialization*/ /*TODOCreateoneandmoveamaxofstuffinit.Putthisinitinthesyslib*/ structbi_record*bootinfo=find_bootinfo(); if(bootinfo)parse_bootinfo(bootinfo); else{/*Loadthebd_tboardinfostructure*/ if(r3)memcpy((void*)&__res,(void*)(r3+KERNELBASE),sizeof(bd_t)); #ifdefCONFIG_BLK_DEV_INITRD /*Loadtheinitrd*/ if(r4){ initrd_start=r4+KERNELBASE; initrd_end=r5+KERNELBASE; } #endif /*Loadthecommandline*/ if(r6){ *(char*)(r7+KERNELBASE)=0; strcpy(cmd_line,(char*)(r6+KERNELBASE)); } } /*PPCSysidentification*/ identify_ppc_sys_by_id(mfspr(SPRN_SVR)); /*BATsetup*/ mpc52xx_set_bat(); /*NoISAbusbydefault*/ isa_io_base=0; isa_mem_base=0; /*Powersave*/ /*Thisisprovidedasanexampleonhowtodoit.ButyouneedtobeawarethatNAPdisable bussnoopandthatmayberequiredforsomedevicestoworkproperly,likeUSB ...*/ /*powersave_nap=1;*/ /*Setuptheppc_mdstruct*/ ppc_md.setup_arch=lite5200_setup_arch; ppc_md.show_cpuinfo=lite5200_show_cpuinfo; ppc_md.show_percpuinfo=NULL; ppc_md.init_IRQ=mpc52xx_init_irq; ppc_md.get_irq=mpc52xx_get_irq; #ifdefCONFIG_PCI ppc_md.pci_map_irq=lite5200_map_irq; #endif ppc_md.find_end_of_memory=mpc52xx_find_end_of_memory; ppc_md.setup_io_mappings=mpc52xx_map_io; ppc_md.restart=mpc52xx_restart; ppc_md.power_off=mpc52xx_power_off; ppc_md.halt=mpc52xx_halt; /*NotimekeeperontheLITE5200*/ ppc_md.time_init=NULL; ppc_md.get_rtc_time=NULL; ppc_md.set_rtc_time=NULL; ppc_md.calibrate_decr=mpc52xx_calibrate_decr; #ifdefCONFIG_SERIAL_TEXT_DEBUG ppc_md.progress=mpc52xx_progress; #endif } This function contains much of the customizing that is required for this particular platform. It starts by searching for board-specific data supplied by the bootloader. We defer discussion of the detailsofthisuntilSection16.3.2,"BoardInformationStructure." Followingthis, ifyourkernel is configured for an initial ramdisk(initrd)[110], thestartandend addresses of the ramdisk image are saved. Notice that they are passed in the PowerPC generalpurpose registers r4 and r5 by convention. It is the bootloader's responsibility to pass the initrd addressesintheseregisters.Later,thekernelwillusetheseaddressestoloadtheinitrdimagefrom rawmemory(wherethebootloaderplacedit,oranonvolatileFlashimage)intoaninternalkernel ramdiskstructure. Nextweseecodetostorethekernelcommandline,whoseaddressispassedintoplatform_init() via registers r6 and r7, marking the start and end addresses, respectively. This differs from the methoddescribedearlierforstoringastatickernelcommandlineinonespecificdetail:thiskernel commandlinewaspassedtoplatform_init()fromthebootloader,asopposedtobeingcompiledinto thekernel. Copying the initrd and kernel command line is very straightforward. Basically, the registers passed in from the bootloader contain the memory addresses where these data structures reside. There is one minor subtlety, however. You may have already wondered about the purpose of the constantKERNELBASE.Understandingthisiskeytograspingoneofthemorecomplexpartsofthe bootsequence. The addresses the bootloader provides are physical addresses. This means they are the real hardwareaddresseswherethedataresidesinthememorychips.Thebootloadertypicallyoperates withoutsupportforvirtualmemory.However,thekernelitselfisstaticallylinkedtoawell-known, user-configuredbaseaddress.ThisaddressisKERNELBASE.(Thevalueitselfisnotrelevanttothe discussionitisuserconfigurablebutvirtuallyneverchangedfromitsdefaultvalueof0xC0000000.) Thissetsupaninterestingsituationinhead.S.Whenthekernelisdecompressedandrelocatedto RAM (usually to location 0), all of its code and data symbols are linked at the kernel's virtual address,KERNELBASE.Thiscanbeseenbyexaminingthekernelsymbolmapfile,producedduring thekernelbuildprocess,System.map.[111]However,theexecutioncontextpriortoenablingtheMMU is such that physical addresses are real hardware addresses. This means that all the code prior to enablingtheMMUandvirtualmemorymappingmustberelocatable,andaccesstosymbolsmustbe fixedup.Thisinvolvesaddinganoffsettothesymbol'saddresstoaccessit.Anexamplewillmake thisclear. 16.3.1.EarlyVariableAccess Let's assume that a code segment very early in the boot process needs to access the variable cmd_linesoearlythatwe'reexecutingin1:1realtophysicalmapping.Aspointedoutearlier,this variableisdefinedinhead.Sandwillendupinthe.datasectionwhenthekernelislinked.Fromthe Linuxkernel'sSystem.mapfile,youcanfindthelinkedaddressesforcmd_line: $catSystem.map|grepcmd_line c0115000Dcmd_line Ifwewererunninginreal=physicalmode(MMUdisabled)andaccessedthisvariableusingits symbol,wewouldbetryingtoreadorwritetoanaddressgreaterthan3GB.Mostsmallerembedded systemsdon'thaveanyRAMinthisregion,andtheresultswouldbeundefinedorwouldresultina crash. Even if we had physical RAM at that address, it is unlikely that it would be mapped and accessiblethisearlyinthebootprocess.Sowehavetoadjustourreferencetothisvariabletoaccess it. Listing16-9reproducesacodesnippetfromhead.Sthatdoesjustthat. Listing16-9.VariableReferenceFixup relocate_kernel: addisr9,r26,klimit@ha/*fetchklimit*/ lwzr25,klimit@l(r9) addisr25,r25,-KERNELBASE@h ThiscodesnippetfromthePowerPChead.Sisagoodexampleoftheissuewearedescribing. Thevariableklimitrepresentstheendofthekernelimage.Itisdefinedelsewhereaschar*klimit. Therefore, it is a pointerit is an address that contains an address. In Listing 16-9, we fetch the address of klimit, sum it with a previously calculated offset that is passed in r26, and deposit the resultingvalueinregisterr9.Registerr9nowcontainsthehigh-order16bitsoftheadjustedaddress of klimit, with the low-order bits zeroed.[112] It was adjusted by the offset value previously calculatedandpassedinregisterr26. Inthenextline,thelwzinstructionusesregisterr9togetherwiththeoffsetofklimit(thelower16 bitsoftheklimitaddress)asaneffectiveaddressfromwhichtoloadregisterr25.(Remember,klimit is a pointer, and we are interested in the value that klimit points to.) Register r25 now holds the pointerthatwasstoredinthevariableklimit.InthefinallineofListing16-9,wesubtractthekernel's linkedbaseaddress(KERNELBASE)fromr25toadjustthepointertoouractualphysicaladdress. InC,itwouldlooklikethis: unsignedint*tmp;/*representsr25*/ tmp=*klimit; tmp-=KERNELBASE; Insummary,wereferencedapointerstoredinklimitandadjusteditsvaluetoourreal(physical) addresssowecanuseitscontents.WhenthekernelenablestheMMUandvirtualaddressing,weno longer have to worry about thisthe kernel will be running at the address where it was linked, regardlessofwhereinphysicalmemoryitisactuallylocated. 16.3.2.BoardInformationStructure Many bootloaders areusedforPowerPCplatforms,butthereisstillnounifiedwaytopassin board-specific data such as serial port baud rate, memory size, and other low-level hardware parameters that the bootloader has configured. The platform-initialization file from Listing 16-8 supportstwodifferentmethods,datastoredasstructbi_recordanddatastoredasstructbd_info.[113] Both methods provide similar results: hardware-specific data is passed from the bootloader to the kernelinthesestructures. From Listing 16-8, here is the code snippet that saves the bootloader-supplied hardware configuration: structbi_record*bootinfo=find_bootinfo(); if(bootinfo)parse_bootinfo(bootinfo); else{ /*Loadthebd_tboardinfostructure*/ if(r3)memcpy((void*)&__res,(void*)(r3+KERNELBASE),sizeof(bd_t)); First,wesearchforaspecialtagthatidentifiesthedatastructureasastructbi_record.Ifthatis found,thebootinfopointerissettotheaddressofthestartofthebootinforecords.Fromthere,the records are parsed and the hardware related data is gathered. This can be seen by inspecting .../arch/ppc/kernel/setup.c.Currently,bi_recordscancontainthekernelcommandline,thestartand end address of the initrd image, the machine type, and the size of the memory. Of course, you can extendthisforyourownrequirements. Ifnobi_recorddataisfound,thePowerPCarchitectureexpectsthisdataintheformofU-Boot board information structure, or bd_info. It is the bootloader's responsibility to construct this data structure and pass the address in register r3. Currently, many bits of hardware information are availableinthebd_infostructure,includinginformationonDRAM,FLASH,SRAM,processorclock rates,busfrequencies,serialportbaudratesetting,andmore. The bi_record structure can be examined in .../include/asm-ppc/bootinfo.h, and the bd_info structurecanbefoundin.../include/asm-ppc/ppcboot.h. Itisthe responsibilityofthe platform-initializationroutinestomakeuseofanyofthedatathat mightbenecessarytocompletethehardwaresetup,ortocommunicateittothekernel.Forexample, platform_init()setsupapointertoafunctionwhosenamerevealsitspurpose.ThecodefromListing 16-8isreproducedhere: ppc_md.find_end_of_memory=mpc52xx_find_end_of_memory; Looking at the function mpc52xx_find_end_of_memory(), .../arch/ppc/syslib/mpc52xx_setup.c,wefindthefollowing: u32ramsize=__res.bi_memsize; if(ramsize==0){ .../*Finditanotherway*/ } which is found in returnramsize; The__resdatastructureaboveistheboardinformationstructure,whoseaddresswaspassedto us from the bootloader in register r3 above. As you can see, the generic setup code stored the residualdata(asitisoftencalled)passedinbythebootloader,butit'suptothemachineorplatformspecificcodetomakeuseofit. 16.3.3.Machine-DependentCalls Many common routines that the kernel needs either for initialization or for operation are architectureandmachine(CPU)dependent.Fromtheplatform_init()functionreproducedinListing 16-8,wesawthefollowing: ... /*Setuptheppc_mdstruct*/ ppc_md.setup_arch=lite5200_setup_arch; ppc_md.show_cpuinfo=lite5200_show_cpuinfo; ppc_md.show_percpuinfo=NULL; ppc_md.init_IRQ=mpc52xx_init_irq; ppc_md.get_irq=mpc52xx_get_irq; #ifdefCONFIG_PCI ppc_md.pci_map_irq=lite5200_map_irq; #endif ppc_md.find_end_of_memory=mpc52xx_find_end_of_memory; ppc_md.setup_io_mappings=mpc52xx_map_io; ppc_md.restart=mpc52xx_restart; ppc_md.power_off=mpc52xx_power_off; ppc_md.halt=mpc52xx_halt; ... Lines similar to these make up the rest of the platform_init() function. Here the bulk of the platform-specificneedsarecommunicatedtotheLinuxkernel.Theglobalvariableppc_md,oftype struct machdep_calls, provides the hooks to easily customize the Linux kernel for a PowerPC platform. This variable is declared in .../arch/ppc/kernel/setup.c. Many places in the PowerPCspecific kernel branch call functions indirectly through this structure. For example, Listing 16-10 reproducesaportionof.../arch/ppc/kernel/setup.c,whichcontainssupportfortherestart,power-off, andhaltfunctions: Listing16-10.GenericPowerPCMachineFunctions voidmachine_restart(char*cmd){ #ifdefCONFIG_NVRAM nvram_sync(); #endif ppc_md.restart(cmd); } voidmachine_power_off(void){ #ifdefCONFIG_NVRAM nvram_sync(); #endif ppc_md.power_off(); } voidmachine_halt(void){ #ifdefCONFIG_NVRAM nvram_sync(); #endif ppc_md.halt(); } Thesefunctionsarecalledviatheppc_mdstructureandcontainthemachine-orplatform-specific variantsofthesefunctions.Youcanseethatsomeofthesefunctionsaremachinespecificandcome from mpc52xx_* variants of the functions. Examples of these include mpc52xx_restart and mpc52xx_map_io. Others are specific to the hardware platform. Examples of platform-specific routinesincludelite5200_map_irqandlite5200_setup_arch. 16.4.PuttingItAllTogether Now that we have a reference from which to proceed, we can create the necessary files and functionsforourowncustomboard.WecopytheLite5200platformfilesforourbaselineandmodify themforourcustomPowerPCplatform.We'llcallournewplatformPowerDNA.Thestepswewill performforthiscustomportareasfollows: 1.Addanewconfigurationoptionto...arch/ppc/Kconfig. 2.Copylite5200.*topowerdna.*asabaseline. 3.Editnewpowerdna.*filesasappropriateforourplatform. 4.Edit.../arch/ppc/Makefiletoconditionallyincludepowerdna.o. 5.Compile,load,anddebug! YoulearnedhowtoaddaconfigurationoptiontoKconfiginChapter4.Theconfigurationoption forournewPowerDNAportisdetailedinListing16-11. Listing16-11.ConfigurationOptionforPowerDNA configPOWERDNA bool"UnitedElectronicsIndustriesPowerDNA" selectPPC_MPC52xx help SupportfortheUEIPowerDNAboard This Kconfig entry is added just below the entry for LITE5200 because they are related.[114] Figure16-4illustratestheresultswhentheconfigurationutilityisinvoked. Figure16-4.MachinetypeoptionforPowerDNA NoticethatwhentheuserselectsPOWERDNA,twoimportantactionsareperformed: 1. The CONFIG_PPC_MPC52xx configuration option is automatically selected. This is accomplishedbytheselectkeywordinListing16-11. 2. A new configuration option, CONFIG_POWERDNA, is defined that will drive the configurationforourbuild. The next step is to copy the files closest to our platform as the basis of our new platforminitializationfiles.WehavealreadydecidedthattheLite5200platformfitsthebill.Copylite5200.c to powerdna.c, and lite5200.h to powerdna.h. The difficult part comes next. Using the hardware specifications, schematics, and any other data you have on the hardware platform, edit the new powerdna.* files as appropriate for your hardware. Get the code to compile, and then proceed to bootanddebugyournewkernel.Thereisnoshortcuthere,noranysubstituteforexperience.Itisthe hardworkofporting,butnowatleastyouknowwheretostart.Manytipsandtechniquesforkernel debuggingarepresentedinChapter14,"KernelDebuggingTechniques." Tosummarizeourportingeffort,Listing16-12detailsthefilesthathavebeenaddedormodified togetLinuxrunningonthePowerDNAboard. Listing16-12.PowerDNANeworModifiedKernelFiles linux-2.6.14/arch/ppc/configs/powerdna_defconfig linux-2.6.14/arch/ppc/Kconfig linux-2.6.14/arch/ppc/platforms/Makefile linux-2.6.14/arch/ppc/platforms/powerdna.c linux-2.6.14/arch/ppc/platforms/powerdna.h linux-2.6.14/drivers/net/fec_mpc52xx/fec.c linux-2.6.14/drivers/net/fec_mpc52xx/fec.h linux-2.6.14/drivers/net/fec_mpc52xx/fec_phy.h linux-2.6.14/include/asm-ppc/mpc52xx.h The first file is the default configuration, which enables a quick kernel configuration based on defaults.Itisenabledbyinvokingmakeasfollows: $makeARCH=ppcCROSS_COMPILE=<cross-prefix>powerdna_defconfig We'vealreadydiscussedthechangestotheKconfigfile.Modificationtothemakefileistrivialthe purpose is to add support for the new kernel configuration based on CONFIG_POWERDNA. The changeconsistsofaddingasingleline: obj-$(CONFIG_POWERDNA)+=powerdna.o Theheartofthechangescomeinthepowerdna.[c|h]filesandchangestotheFEC(FastEthernet Controller) layer. There were minor differences between powerdna.c and lite5200.c, the file from whichitwasderived.Twoprimaryissuesrequiredchanges.First,PCIwasdisabledbecauseitis not used in the PowerDNA design. This required some minor tweaking. Second, the PowerDNA design incorporates an unmanaged Ethernet physical-layer chip that required slight changes in the hardwaresetupandtheFEClayer.Thisworkconstitutedthemajorityoftheportingeffort.Thepatch fileconsistsof1120lines,butthebulkofthoselinesarethedefaultconfiguration,whichisonlya convenienceforthedeveloperandisnotstrictlynecessary.Removingthat,thepatchreducesto411 lines. 16.4.1.OtherArchitectures Weexaminedthedetailsofhowagivenplatformfitsintothekernel,andthefacilitiesthatexist forportingtoanewboard.Ourreferenceforthischapterandthediscussionswithincamefromthe PowerPCarchitecturebranchofthekernel.Theotherarchitecturesdifferinmanydetailedaspectsof how various hardware platforms are incorporated, but the concepts are similar. When you have learnedhowtonavigateasinglearchitecture,youhavetheknowledgeandtoolstolearnthedetails oftheotherarchitectures. 16.5.ChapterSummary • Porting Linux to a custom board based on a Linux-supported CPU can be relatively straightforward.ThereisnosubstituteforexperienceandknowledgeoftheLinuxcodebaseandyour hardwareplatform. • Starting from a working reference configuration based on a hardware platform already supportedprovidesanexcellentbasisforyourownmodifications. •Understandingtheflowofinitializationcodeisthekeytoaneasyportingeffort.Wemadeevery effort to leave all generic kernel code untouched and to modify only those files necessary for the platform itself. A significant part of this chapter is devoted to this early flow of control related to platforminitialization. • Make doubly certain that your low-level hardware platform initialization is correct before proceeding. If you find yourself debugging in some obscure part of the Linux slab allocator, for example,it'sagoodbetyou'vemessedsomethingupwithyourhardwarememoryinitialization. • This chapter focused primarily on the PowerPC architecture branch of the Linux kernel. Learningthedetailsofonearchitecturepavesthewayforunderstandingtherest. 16.5.1.SuggestionsforAdditionalReading ProgrammingEnvironmentsManualfor32-BitImplementationsofthePowerPCArchitecture MPCFPE32B/AD12/2001REV2 FreescaleSemiconductor,Inc. MPC5200User'sGuide MPC5200UGRev301/22005 FreescaleSemiconductor,Inc. Chapter17.LinuxandRealTime WhenLinuxbeganlifeonanInteli386processor,nooneeverexpectedthesuccessLinuxwould enjoy in server applications. That success has led to Linux being ported to many different architectures and used by developers for embedded systems from cellular handsets to telecommunicationsswitches.Notlongago,ifyourapplicationhadreal-timerequirements,youmight nothaveincludedLinuxamongthechoicesforyouroperatingsystem.Thathasallchangedwiththe developmentsinreal-timeLinuxdriven,inlargepart,byaudioandmultimediaapplications. In this chapter, we start with a brief look at the historical development of real-time Linux features.Thenwelookatthefacilitiesavailabletothereal-timeprogrammerandhowthesefacilities areused. 17.1.WhatIsRealTime? Askfivepeoplewhat"realtime"means,and,chancesare,youwillgetfivedifferentanswers. Somemightevencitesomenumbers.Forthepurposesofthediscussiontofollow,wediscusssome scenariosandthenproposeadefinition.Manyrequirementsmightbesaidtobesoftrealtime,while othersarecalledhardrealtime. 17.1.1.SoftRealTime Most agree that soft real time means that the operation has a deadline, but if the deadline is missed,thequalityoftheexperiencecouldbediminished(butnotfatal).Yourdesktopworkstationis aperfectexampleofsoftreal-timerequirements.Whenyouareeditingadocument,youexpecttosee theresultsofyourkeystrokesimmediatelyonthescreen.Whenplayingyourfavorite.mp3file,you expecttohavehigh-qualityaudiowithoutanyclicks,pops,orgapsinthemusic. Ingeneralterms,humanscannotseeorheardelaysbelowafewtensofmilliseconds.Ofcourse, themusiciansinthecrowdwilltellyouthatmusiccanbecoloredbydelayssmallerthanthat.Ifa deadlineismissedbytheseso-calledsoftreal-timeevents,theresultsmaybeundesirable,leadingto alowerlevelof"quality"oftheexperience,butnotcatastrophic. 17.1.2.HardRealTime Hardrealtimeischaracterizedbytheresultsofamisseddeadline.Inahardreal-timesystem,if adeadlineismissed,theresultsareoftencatastrophic.Ofcourse,catastrophicisarelativeterm.If your embedded device is controlling the fuel flow to a jet aircraft engine, missing a deadline to respondtopilotinputorachangeinoperationalcharacteristicscanleadtodisastrousresults. Notethatthedurationofthedeadlinehasnobearingonthereal-timecharacteristic.Servicingthe tick on an atomic clock is such an example. As long as the tick is processed within the 1-second windowbeforethenexttick,thedataremainsvalid.Missingtheprocessingonatickmightthrowoff ourglobalpositioningsystemsbyfeetorevenmiles! Withthisinmind,wedrawonacommonlyusedsetofdefinitionsforsoftandhardrealtime.For softreal-timesystems,thevalueofacomputationorresultisdiminishedifadeadlineismissed.For hardreal-timesystems,ifasingledeadlineismissed,thesystemisconsideredtohavefailed,and mighthavecatastrophicconsequences. 17.1.3.LinuxScheduling UNIX and Linux were both designed for fairness in their process scheduling. That is, the scheduler tries its best to allocate available resources across all processes that need the CPU and guarantee each process that they can make progress. This very design objective is counter to the requirementforareal-timeprocess.Areal-timeprocessmustrunassoonaspossibleafteritisready torun.Realtimemeanshavingpredictableandrepeatablelatency. 17.1.4.Latency Real-timeprocessesareoftenassociatedwithaphysicalevent,suchasaninterruptarrivingfrom a peripheral device. Figure 17-1 illustrates the latency components in a Linux system. Latency measurementbeginsuponreceiptoftheinterruptwewanttoprocess.Thisisindicatedbytimet0in Figure 17-1. Sometime later, the interrupt is taken and control is passed to the Interrupt Service Routine(ISR).Thisisindicatedbytimet1.Thisinterruptlatencyisalmostentirelydictatedbythe maximum interruptofftime[115] the time spent in a thread of execution that has hardware interrupts disabled. Figure17-1.Latencycomponents It is considered good design practice to minimize the processing done in the actual interrupt service routine. Indeed, this execution context is limited in capability (for example, an ISR cannot call a blocking function, one that might sleep), so it is desirable to simply service the hardware deviceandleavethedataprocessingtoaLinuxbottomhalf,[116]alsocalledsoftIRQs. WhentheISR/bottomhalfhasfinisheditsprocessing,theusualcaseistowakeupauserspace processthatiswaitingforthedata.Thisisindicatedbytimet2inFigure17-1.Atsomepointintime later,thereal-timeprocessisselectedbytheschedulertorunandisgiventheCPU.Thisisindicated by time t3 in Figure 17-1. Scheduling latency is affected primarily by the number of processes waitingfortheCPUandtheprioritiesamongthem.SettingtheRealTimeattributeonaprocessgives ithigherpriorityovernormalLinuxprocessesandallowsittobethenextprocessselectedtorun, assuming that it is the highest priority real-time process waiting for the CPU. The highest-priority real-timeprocessthatisreadytorun(notblockedonI/O)willalwaysrun.You'llseehowtosetthis attributeshortly. 17.2.KernelPreemption IntheearlyLinuxdaysofLinux1.x,therewasnokernelpreemption.Thismeantthatwhenauser spaceprocessrequestedkernelservices,noothertaskcouldbescheduledtorununtilthatprocess either blocked (goes to sleep) waiting on something (usually I/O), or until the kernel request is completed.Makingthekernelpreemptable[117]meansthatwhileoneprocessisrunninginthekernel, another process can preempt the first and be allowed to run even though the first process had not completeditsin-kernelprocessing.Figure17-2illustratesthis. Figure17-2.Kernelpreemption Inthisfigure,ProcessAhasenteredthekernelviaasystemcall.Perhapsitwasacalltowrite() to a device such as the console or a file. While executing in the kernel on behalf of Process A, Process B with higher priority is woken up by an interrupt. The kernel preempts Process A and assignstheCPUtoProcessB,eventhoughProcessAhadneitherblockednorcompleteditskernel processing. 17.2.1.ImpedimentstoPreemption Thechallengeinmakingthekernelfullypreemptableistoidentifyalltheplacesinthekernelthat must be protected from preemption. These are the critical sections within the kernel where preemption cannot be allowed to occur. For example, assume that Process A in Figure 17-2 is executing in the kernel performing a file system operation. At some point, the code might need to writetoanin-kerneldatastructurerepresentingafileonthefilesystem.Toprotectthatdatastructure from corruption, the process must lock out all other processes from accessing the shared data structure.Listing17-1illustratesthisconceptusingCsyntax. Listing17-1.LockingCriticalSections ... preempt_disable(); ... /*Criticalsection*/ update_shared_data(); ... preempt_enable(); ... Ifwe did notprotectshareddatainthisfashion,theprocessupdatingtheshareddatastructure could be preempted in the middle of the update. If another process attempted to update the same shared data, corruption of the data would be virtually certain. The classic example is when two processesareoperatingdirectlyoncommonvariablesandmakingdecisionsontheirvalues.Figure 17-3illustratessuchacase. Figure17-3.Shareddataconcurrencyerror In Figure 17-3, Process A is interrupted after updating the shared data but before it makes a decision based on it. By design, Process A cannot detect that it has been preempted. Process B changesthevalueoftheshareddatabeforeProcessAgetstorunagain.Asyoucansee,ProcessA willbemakingadecisionbasedonavaluedeterminedbyProcessB.Ifthisisnotthebehavioryou seek,youmustdisablepreemptioninProcessAaroundtheshareddatainthiscase,theoperationand decisiononthevariablecount. 17.2.2.PreemptionModels The first solution to kernel preemption was to place checks at strategic locations within the kernel code where it was known to be safe to preempt the current thread of execution. These locations included entry and exit to system calls, release of certain kernel locks, and return from interrupt processing. At each of these points, code similar to Listing 17-2 was used to perform preemption. Listing17-2.CheckforPreemptionalaLinux2.4+PreemptPatch ... /* *Thiscodeisexecutedatstrategiclocationswithin *theLinuxkernelwhereitisknowntobesafeto *preemptthecurrentthreadofexecution */ if(kernel_is_preemptable()&¤t->need_resched)preempt_schedule(); ... /* *Thiscodeisin.../kernel/sched.candisinvokedfrom *thosestrategiclocationsasabove */ #ifdefCONFIG_PREEMPT asmlinkagevoidpreempt_schedule(void){ while(current->need_resched){ ctx_sw_off(); current->state|=TASK_PREEMPTED; schedule(); current->state&=~TASK_PREEMPTED; ctx_sw_on_no_preempt(); } } #endif ... The first snippet of code in Listing 17-2 (simplified from the actual code) is invoked at those strategiclocationsdescribedearlier,whereitisknownthatthekernelissafetopreempt.Thesecond snippet of codeinListing17-2istheactualcodefromanearlyLinux2.4kernelwiththepreempt patchapplied.Thisinterestingwhileloopcausesacontextswitchviathecalltoschedule()untilall requestsforpreemptionhavebeensatisfied. Although this approach led to reduced latencies in the Linux system, it was not ideal. The developers working on low-latency soon realized the need to "flip the logic." With earlier preemptionmodels,wehadthis: •TheLinuxkernelwasfundamentallynonpreemptable. •Preemptionchecksweresprinkledaroundthekernelatstrategiclocationsknowntobesafefor preemption. •Preemptionwasenabledonlyattheseknown-safepoints. Toachieveafurthersignificantreductioninlatency,wewantthisinapreemptablekernel: •TheLinuxkernelisfullypreemptableeverywhere. •Preemptionisdisabledonlyaroundcriticalsections. This is where the kernel developers have been heading since the original preemptable kernel patch series. However, this is no easy task. It involves poring over the entire kernel source code base,analyzingexactlywhatdatamustbeprotectedfromconcurrency,anddisablingpreemptionat only those locations. The method used for this has been to instrument the kernel for latency measurements,findthelongestlatencycodepaths,andfixthem.ThemorerecentLinux2.6kernels can be configured for very low-latency applications because of the effort that has gone into this "lock-breaking"methodology. 17.2.3.SMPKernel It is interesting to note that much of the work involved in creating an efficient multiprocessor architecture also benefits real time. The SMP challenge is more complex than the uniprocessor challenge because there is an additional element of concurrency to protect against. In the uniprocessor model, only a single task can be executing in the kernel at a time. Protection from concurrency involves only protection from interrupt or exception processing. In the SMP model, multiple threads of execution in the kernel are possible in addition to the threat from interrupt and exceptionprocessing. SMPhasbeensupportedfromearlyLinux2.xkernels.ABigKernelLock(BKL)wasusedto protect against concurrency in the transition from uniprocessor to SMP operation. The BKL is a global spinlock, which prevents any other tasks from executing in the kernel. In his excellent book Linux Kernel Development (Novell Press, 2005), Robert Love characterized the BKL as the "redheaded stepchild of the kernel." In describing the characteristics of the BKL, Robert jokingly added"evil"toitslistofattributes! Early implementations of the SMP kernel based on the BKL led to significant inefficiencies in scheduling.ItwasfoundthatoneoftheCPUscouldbekeptidleforlongperiodsoftime.Muchofthe work that led to an efficient SMP kernel also directly benefited real-time applicationsprimarily loweredlatency.ReplacingtheBKLwithsmaller-grainedlockingsurroundingonlytheactualshared datatobeprotectedledtosignificantlyreducedpreemptionlatency. 17.2.4.SourcesofPreemptionLatency A real-time system must be capable of servicing its real-time tasks within a specified upper boundary of time. Achieving consistently low preemption latency is critical to a real-time system. Thetwosinglelargestcontributorstopreemptionlatencyareinterrupt-contextprocessingandcritical sectionprocessingwhereinterruptsaredisabled.Youhavealreadylearnedthatagreatdealofeffort has been targeted at reducing the size (and thus, duration) of the critical sections. This leaves interrupt-context processingasthenextchallenge.ThiswasansweredwiththeLinux 2.6 real-time patch. 17.3.Real-TimeKernelPatch Supportforhardrealtimeisnotinthemainlinekernel.orgsourcetree.Toenablehardrealtime, apatchmustbeapplied.Thereal-timekernelpatchisthecumulativeresultofseveralinitiativesto reduceLinuxkernellatency.Thepatchhadmanycontributors,anditiscurrentlymaintainedbyIngo Molnar;youcanfinditathttp://people.redhat.com/~mingo/realtime-preempt.Thesoftreal-timeperformanceofthe 2.6Linuxkernelhasimprovedsignificantlysincetheearly2.6kernelreleases.When2.6wasfirst released, the 2.4 Linux kernel was substantially better in soft real-time performance. Since about Linux 2.6.12, soft real-time performance in the single-digit milliseconds on a reasonably fast x86 processor is readily achieved. To get repeatable performance beyond this requires the real-time patch. Thereal-timepatchaddsseveralimportantfeaturestotheLinuxkernel.Figure17-4displaysthe configurationoptionsforPreemptionmodewhenthereal-timepatchhasbeenapplied. Figure17-4.Preemptionmodeswithreal-timepatch The real-time patch adds a fourth preemption mode called PREEMPT_RT, or Preempt Real Time.Thefourpreemptionmodesareasfollows: •PREEMPT_NONE:Noforcedpreemption.Overalllatencyis,onaverage,good,buttherecan besomeoccasionallongdelays.Bestsuitedforapplicationsforwhichoverallthroughputisthetop designcriteria. • PREEMPT_VOLUNTARY : First stage of latency reduction. Additional explicit preemption points are placed at strategic locations in the kernel to reduce latency. Some loss of overall throughputistradedforlowerlatency. •PREEMPT_DESKTOP:Thismodeenablespreemptioneverywhereinthekernelexceptwhen processingwithincriticalsections.Thismodeisusefulforsoftreal-timeapplicationssuchasaudio andmultimedia.Overallthroughputistradedforfurtherreductionsinlatency. • PREEMPT_RT : Features from the real-time patch are added, including replacing spinlocks withpreemptablemutexes.Thisenablesinvoluntarypreemptioneverywherewithinthekernelexcept forthoseareasprotectedbypreempt_disable().Thismodesignificantlysmoothesoutthevariationin latency(jitter)andallowsalowandpredictablelatencyfortime-criticalreal-timeapplications. Ifkernelpreemptionisenabledinyourkernelconfiguration,itcanbedisabledatboottimeby addingthefollowingkernelparametertothekernelcommandline: preempt=0 17.3.1.Real-TimeFeatures SeveralnewLinuxkernelfeaturesareenabledwithCONFIG_PREEMPT_RT.FromFigure17-4, we see several new configuration settings. These and other features of the real-time Linux kernel patcharedescribedhere. 17.3.1.1.SpinlockConvertedtoMutex The real-time patch converts most spinlocks in the system to mutexes. This reduces overall latencyatthecostofslightlyreducedthroughput.Thebenefitofconvertingspinlockstomutexesis thattheycanbepreempted.IfProcessAisholdingalock,andProcessBatahigherpriorityneeds thesamelock,ProcessAcanpreemptProcessBinthecasewhereitisholdingamutex. 17.3.1.2.ISRsasKernelTasks WithCONFIG_PREEMPT_HARDIRQselected,interruptserviceroutines[118](ISRs)areforced to run in process context. This gives the developer control over the priority of ISRs because they become schedulable entities. As such, they also become preemptable to allow higher-priority hardwareinterruptstobehandledfirst. This is a powerful feature. Some hardware architectures do not enforce interrupt priorities. Thosethatdomightnotenforcetheprioritiesconsistentwithyourspecifiedreal-timedesigngoals. UsingCONFIG_PREEMPT_HARDIRQ,youarefreetodefinetheprioritiesatwhicheachIRQwill run. ConversionofISRstothreadscanbedisabledatruntimethroughthe/procfilesystemoratboot timebyenteringaparameteronthekernelcommandline.Whenenabledintheconfiguration,unless youspecifyotherwise,ISRthreadingisenabledbydefault. TodisableISRthreadingatruntime,issuethefollowingcommandasroot: #echo'0'>/proc/sys/kernel/hardirq_preemption Toverifythesetting,displayitasfollows: #cat/proc/sys/kernel/hardirq_preemption 1 TodisableISRthreadingatboottime,addthefollowingparametertothekernelcommandline: hardirq-preempt=0 17.3.1.3.PreemptableSoftirqs CONFIG_PREEMPT_SOFTIRQ reduces latency by running softirqs within the context of the kernel's softirq daemon (ksoftirqd). ksoftirqd is a proper Linux task (process). As such, it can be prioritized and scheduled along with other tasks. If your kernel is configured for real time, and CONFIG_PREEMPT_SOFTIRQisenabled,theksoftirqdkerneltaskiselevatedtoreal-timepriority tohandlethesoftirqprocessing.[119]Listing17-3showsthecoderesponsibleforthisfromarecent Linuxkernel,foundin.../kernel/softirq.c. Listing17-3.PromotingksoftirqtoReal-TimeStatus staticintksoftirqd(void*__bind_cpu){ structsched_paramparam={.sched_priority=24}; printk("ksoftirqdstartedup.\n"); #ifdefCONFIG_PREEMPT_SOFTIRQS printk("softirqRTprio:%d.\n",param.sched_priority); sys_sched_setscheduler(current->pid,SCHED_FIFO,¶m); #else set_user_nice(current,-10); #endif ... HereweseethatifCONFIG_PREEMPT_SOFTIRQSisenabledinthekernelconfiguration,the ksoftirqd kernel task is promoted to a real-time task (SCHED_FIFO) at a real-time priority of 24 usingthesys_sched_setscheduler()kernelfunction. SoftIRQthreadingcanbedisabledatruntimethroughthe/procfilesystem,aswellasthroughthe kernelcommandlineatboottime.Whenenabledintheconfiguration,unlessyouspecifyotherwise, SoftIRQthreadingisenabledbydefault.TodisableSoftIRQthreadingatruntime,issuethefollowing commandasroot: #echo'0'>/proc/sys/kernel/softirq_preemption Toverifythesetting,displayitasfollows: #cat/proc/sys/kernel/softirq_preemption 1 TodisableSoftIRQthreadingatboottime,addthefollowingparametertothekernelcommand line: softirq-preempt=0 17.3.1.4.PreemptRCU RCU(Read-Copy-Update)[120]isaspecialformofsynchronizationprimitiveintheLinuxkernel designed for data that is read frequently but updated infrequently. You can think of RCU as an optimized reader lock. The real-time patch adds CONFIG_PREEMPT_RCU, which improves latencybymakingcertainRCUsectionspreemptable. 17.3.2.O(1)Scheduler TheO(1)schedulerhasbeenaroundsincethedaysofLinux2.5.Itismentionedherebecauseitis acriticalcomponentofareal-timesolution.TheO(1)schedulerisasignificantimprovementover the previous Linux scheduler. It scales better for systems with many processes and helps produce loweroveralllatency. Incaseyouarewondering,O(1)isamathematicaldesignationforasystemofthefirstorder.In this context, it means that the time it takes to make a scheduling decision is not dependent on the numberofprocessesonagivenrunqueue.TheoldLinuxschedulerdidnothavethischaracteristic, anditsperformancedegradedwiththenumberofprocesses.[121] 17.3.3.CreatingaReal-TimeProcess Youcandesignateaprocessasrealtimebysettingaprocessattributethattheschedulerusesas partofitsschedulingalgorithm.Listing17-4showsthegeneralmethod. Listing17-4.CreatingaReal-TimeProcess #include<sched.h> #defineMY_RT_PRIORITYMAX_USER_RT_PRIO/*Highestpossible*/ intmain(intargc,char**argv){ ... intrc,old_scheduler_policy; structsched_parammy_params; ... /*Passingzerospecifiescaller's(our)policy*/ old_scheduler_policy=sched_getscheduler(0); my_params.sched_priority=MY_RT_PRIORITY; /*Passingzerospecifiescallers(our)pid*/ rc=sched_setscheduler(0,SCHED_RR,&my_params); if(rc==-1)handle_error(); ... } Thiscodesnippetdoestwothingsinthecalltosched_setscheduler().Itchangesthescheduling policytoSCHED_RRandraisesitsprioritytothemaximumpossibleonthesystem.Linuxsupports threeschedulingpolicies: •SCHED_OTHER:NormalLinuxprocess,fairnessscheduling •SCHED_RR:Real-timeprocesswithatimeslicethatis,ifitdoesnotblock,itisallowedto runforagivenperiodoftimedeterminedbythescheduler • SCHED_FIFO : Real-time process that runs until it either blocks or explicitly yields the processor,oruntilanotherhigher-prioritySCHED_FIFOprocessbecomesrunnable The man page for sched_setscheduler provides more detail on the three different scheduling policies. 17.3.4.CriticalSectionManagement Whenwritingkernelcode,suchasacustomdevicedriver,youwillencounterdatastructuresthat you must protect from concurrent access. The easiest way to protect critical data is to disable preemptionaroundthecriticalsection.Keepthecriticalpathasshortaspossibletomaintainalow maximumlatencyforyoursystem.Listing17-5showsanexample. Listing17-5.ProtectingCriticalSectioninKernelCode ... /* *Declareandinitializeagloballockforyour *criticaldata */ DEFINE_SPINLOCK(my_lock); ... intoperate_on_critical_data(){ ... spin_lock(&my_lock); ... /*Updatecritical/shareddata*/ ... spin_unlock(&my_lock); ... } Whenatasksuccessfullyacquiresaspinlock,preemptionisdisabledandthetaskthatacquired the spinlock is allowed into the critical section. No task switches can occur until a spin_unlock operationtakesplace.Thespin_lock()functionisactuallyamacrothathasseveralforms,depending onthekernelconfiguration.Theyaredefinedatthetoplevel(architecture-independentdefinitions)in .../include/linux/spinlock.h.Whenthekernelispatchedwiththereal-timepatch,thesespinlocksare promotedtomutexestoallowpreemptionofhigher-priorityprocesseswhenaspinlockisheld. Becausethereal-timepatchislargelytransparenttothedevicedriverandkerneldeveloper,the familiar constructs can be used to protect critical sections, as described in Listing 17-5. This is a major advantage of the real-time patch for real-time applications; it preserves the well-known semanticsforlockingandinterruptserviceroutines. Using the macro DEFINE_SPINLOCK as in Listing 17-5 preserves future compatibility. These macrosaredefinedin.../include/linux/spinlock_types.h. 17.4.DebuggingtheReal-TimeKernel Several configuration options facilitate debugging and performance analysis of the real-time patchedkernel.Theyaredetailedinthefollowingsubsections. 17.4.1.SoftLockupDetection To enable soft lockup detection, enable CONFIG_DETECT_SOFTLOCKUP in the kernel configuration.Thisfeatureenablesthedetectionoflongperiodsofrunninginkernelmodewithouta context switch. This feature exists in non-real-time kernels but is useful for detecting very high latency paths or soft deadlock conditions. To use it, simply enable the feature and watch for any reportsontheconsoleorsystemlog.Reportswillbeemittedsimilartothis: BUG:softlockupdetectedonCPU0 Whenthismessageisemittedbythekernel,itisusuallyaccompaniedbyabacktraceandother information such as the process name and PID. It will look similar to a kernel oops message complete with processor registers. See .../kernel/softlockup.c for details. This information can be usedtohelptrackdownthesourceofthelockupcondition. 17.4.2.PreemptionDebugging To enable preemption debugging, enable CONFIG_DEBUG_PREEMPT in the kernel configuration.Thisdebugfeatureenablesthedetectionofunsafeuseofpreemptionsemanticssuchas preemption count underflows and attempts to sleep while in an invalid context. To use it, simply enablethefeatureandwatchforanyreportsontheconsoleorsystemlog.Hereisjustasmallsample ofreportspossiblewhenpreemptiondebuggingisenabled: BUG:<me><mypid>,possiblewake_upraceon<proc><pid> BUG:lockrecursiondeadlockdetected!<moreinfo> BUG:nonzerolockcount<n>atexittime? Manymoremessagesarepossiblethesearejustafewexamplesofthekindsofproblemsthatcan be detected. These messages will help you avoid deadlocks and other erroneous or dangerous programmingsemanticswhenusingreal-timekernelfeatures.Formoredetailsonthemessagesand conditionsunderwhichtheyareemitted,browsetheLinuxkernelsourcefile.../kernel/rt-debug.c. 17.4.3.DebugWakeupTiming Toenablewakeuptiming,enableCONFIG_WAKEUP_TIMINGinthekernelconfiguration.This debugoptionenablesmeasurementofthetimetakenfromwakingupahigh-priorityprocesstowhen itisscheduledonaCPU.Usingitissimple.Whenconfigured,measurementisdisabled.Toenable themeasurement,dothefollowingasroot: #echo'0'>/proc/sys/kernel/preempt_max_latency Whenthis/procfileissettozero,eachsuccessivemaximumwakeuptimingresultiswrittento thisfile.Toreadthecurrentmaximum,simplydisplaythevalue: #cat/proc/sys/kernel/preempt_max_latency 84 As long as any of the latency-measurement modes are enabled in the kernel configuration, preempt_max_latencywillalwaysbeupdatedwiththemaximumlatencyvalue.Itcannotbedisabled. Writing 0 to this /proc variable simply resets the maximum to zero to restart the cumulative measurement. 17.4.4.WakeupLatencyHistory To enable wakeup latency history, enable CONFIG_WAKEUP_LATENCY_HIST while CONFIG_WAKEUP_TIMING is also enabled. This option dumps all the wakeup timing measurementsenabledbyCONFIG_WAKEUP_TIMINGintoafileforlateranalysis.Anexampleof thisfileanditscontentsispresentedshortlywhenweexamineinterruptoffhistory. • CRITICAL_PREEMPT_TIMING : Measures the time spent in critical sections with preempt disabled. • PREEMPT_OFF_HIST : Similar to WAKEUP_LATENCY_HIST. Gathers preempt off timing measurementsintoabinforlateranalysis. 17.4.5.InterruptOffTiming To enable measurement of maximum interrupt off timing, configure your kernel with CRITICAL_IRQSOFF_TIMING enabled. This option measures time spent in critical sections with irqs disabled. This feature works in the same way as wakeup latency timing. To enable the measurement,dothefollowingasroot: #echo'0'>/proc/sys/kernel/preempt_max_latency Whenthis/procfileissettozero,eachsuccessivemaximuminterruptofftimingresultiswritten tothisfile.Toreadthecurrentmaximum,simplydisplaythevalue: #cat/proc/sys/kernel/preempt_max_latency 97 Youwillnoticethatthelatencymeasurementsforbothwakeuplatencyandinterruptofflatency are enabled and displayed using the same /proc file. This means, of course, that only one measurement can be configured at a time, or the results might not be valid. Because these measurementsaddsignificantruntimeoverhead,itisn'twisetoenablethemallatonceanyway. 17.4.6.InterruptOffHistory Enabling INTERRUPT_OFF_HIST provides functionality similar to that with WAKEUP_LATENCY_HIST. This option gathers interrupt off timing measurements into a file for lateranalysis.Thisdataisformattedasahistogram,withbinsrangingfrom0microsecondstojust over 10,000 microseconds. In the example just given, we saw that the maximum latency was 97 microseconds from that particular sample. Therefore, we can conclude that the latency data in histogramformwillnotcontainanyusefulinformationbeyondthe97-microsecondbin. Historydataisobtainedbyreadingaspecial/procfile.Thisoutputisredirectedtoaregularfile foranalysisorplottingasfollows: #cat/proc/latency_hist/interrupt_off_latency/CPU0>hist_data.txt Listing17-6displaysthefirst10linesofthehistorydata. Listing17-6.InterruptOffLatencyHistory(Head) $cat/proc/latency_hist/interrupt_off_latency/CPU0|head #Minimumlatency:0microseconds. #Averagelatency:1microseconds. #Maximumlatency:97microseconds. #Totalsamples:60097595 #Thereare0samplesgreaterorequalthan10240microseconds #usecssamples 013475417 138914907 22714349 3442308 ... FromListing17-6wecanseetheminimumandmaximumvalues,theaverageofallthevalues, andthetotalnumberofsamples.Inthiscase,weaccumulatedslightlymorethan60millionsamples. Thehistogramdatafollowsthesummaryandcontainsuptoaround10,000bins.Wecaneasilyplot thisdatausinggnuplotasshowninFigure17-5. Figure17-5.Interruptofflatencydata 17.4.7.LatencyTracing TheLATENCY_TRACEconfigurationoptionenablesgenerationofkerneltracedataassociated withthelastmaximumlatencymeasurement.Itisalsomadeavailablethroughthe/procfilesystem.A latency trace can help you isolate the longest-latency code path. For each new maximum latency measurement,anassociatedtraceisgeneratedthatfacilitatestracingthecodepathoftheassociated maximumlatency. Listing 17-7 reproduces an example trace for a 78-microsecond maximum. As with the other measurementtools,enablethemeasurementbywritinga0to/proc/sys/kernel/preempt_max_latency. Listing17-7.InterruptOffMaximumLatencyTrace $cat/proc/latency_trace preemptionlatencytracev1.1.5on2.6.14-rt-intoff-tim_trace ------------------------------------------------------------latency:78us,#50/50,CPU#0|(M:rtVP:0,KP:0,SP:1HP:1) ----------------|task:softirq-timer/0-3(uid:0nice:0policy:1rt_prio:1) ----------------_------=>CPU# /_-----=>irqs-off |/_----=>need-resched ||/_---=>hardirq/softirq |||/_--=>preempt-depth ||||/ |||||delay cmdpid|||||time|caller \/|||||\|/ cat-66370D...1us:common_interrupt((0)) cat-66370D.h.2us:do_IRQ(c013d91c00) cat-66370D.h13us+:mask_and_ack_8259A(__do_IRQ) cat-66370D.h110us:redirect_hardirq(__do_IRQ) cat-66370D.h.12us:handle_IRQ_event(__do_IRQ) cat-66370D.h.13us:timer_interrupt(handle_IRQ_event) cat-66370D.h.15us:handle_tick_update(timer_interrupt) cat-66370D.h116us:do_timer(handle_tick_update) ...<we'reinthetimerinterruptfunction> cat-66370D.h.22us:run_local_timers(update_process_times) cat-66370D.h.22us:raise_softirq(run_local_timers) cat-66370D.h.23us:wakeup_softirqd(raise_softirq) ...<softirqworkpending-needtopreemptissignaled> cat-66370Dnh.34us:wake_up_process(wakeup_softirqd) cat-66370Dnh.35us+:rcu_pending(update_process_times) cat-66370Dnh.39us:scheduler_tick(update_process_times) cat-66370Dnh.39us:sched_clock(scheduler_tick) cat-66370Dnh141us:task_timeslice(scheduler_tick) cat-66370Dnh.42us+:preempt_schedule(scheduler_tick) cat-66370Dnh145us:note_interrupt(__do_IRQ) cat-66370Dnh145us:enable_8259A_irq(__do_IRQ) cat-66370Dnh147us:preempt_schedule(enable_8259A_irq) cat-66370Dnh.48us:preempt_schedule(__do_IRQ) cat-66370Dnh.48us:irq_exit(do_IRQ) cat-66370Dn..49us:preempt_schedule_irq(need_resched) cat-66370Dn..50us:__schedule(preempt_schedule_irq) ...<hereisthecontextswitchtosoftirqd-timerthread> <...>-30D..274us+:__switch_to(__schedule) <...>-30D..276us:__schedule<cat-6637>(7462) <...>-30D..277us:__schedule(schedule) <...>-30D..278us:trace_irqs_on(__schedule) ...<outputtruncatedhereforbrevity> We have trimmed this listing significantly for clarity, but the key elements of this trace are obvious. This trace resulted from a timer interrupt. In the hardirq thread, little is done beyond queuingupsomeworkforlaterinasoftirqcontext.Thisisseenbythewakeup_softirqd()functionat 23 microseconds and is typical for interrupt processing. This triggers the need_resched flag, as showninthetracebytheninthethirdcolumnofthesecondfield.At49microseconds,aftersome processinginthetimersoftirq,theschedulerisinvokedforpreemption.At74microseconds,control ispassedtotheactualsoftirqd-timer/0threadrunninginthisparticularkernelasPID3.(Theprocess namewastruncatedtofitthefieldwidthandisshownas<...>.) Most of the fields of Listing 17-7 have obvious meanings. The irqs-off field contains a D for sectionsofcodewhereinterruptsareoff.Becausethislatencytraceisaninterruptsofftrace,wesee this indicated throughout the trace. The need_resched field mirrors the state of the kernel's need_reschedflag.Annindicatesthattheschedulershouldberunatthesoonestopportunity,anda period(.)meansthatthisflagisnotactive.Thehardirq/softirqfieldindicatesathreadofexecutionin hardirqcontextwithh,andsoftirqcontextwiths.Thepreempt-depthfieldindicatesthevalueofthe kernel'spreempt_countvariable,anindicatorofnestingleveloflockswithinthekernel.Preemption canoccuronlywhenthisvariableisatzero. 17.4.8.DebuggingDeadlockConditions The DEBUG_DEADLOCKS kernel configuration option enables detection and reporting of deadlock conditions associated with the semaphores and spinlocks in the kernel. When enabled, potentialdeadlockconditionsarereportedinafashionsimilartothis: ========================================== [BUG:lockrecursiondeadlockdetected!| -----------------------------------------... Muchinformationisdisplayedafterthebannerlineannouncingthedeadlockdetection,including thelockdescriptor,lockname(ifavailable),lockfileandname(ifavailable),lockowner,whois currentlyholdingthelock,andsoon.Usingthisdebugtool,itispossibletoimmediatelydetermine theoffendingprocesses.Ofcourse,fixingitmightnotbesoeasy! 17.4.9.RuntimeControlofLockingMode The DEBUG_RT_LOCKING_MODE option enables a runtime control to switch the real-time mutexbackintoanonpreemptablemode,effectivelychangingthebehaviorofthereal-time(spinlocks asmutexes)kernelbacktoaspinlock-basedkernel.Aswiththeotherconfigurationoptionswehave covered here, this tool should be considered a development aid to be used only in a development environment. Itdoesnotmakesensetoenableallofthesedebugmodesatonce.Asyoumightimagine,mostof thesedebugmodesaddsizeandsignificantprocessingoverheadtothekernel.Theyaremeanttobe usedasdevelopmentaidsandshouldbedisabledforproductioncode. 17.5.ChapterSummary •Linuxisincreasinglybeingusedinsystemswherereal-timeperformanceisrequired.Examples includemultimediaapplicationsandrobot,industrial,andautomotivecontrollers. • Real-time systems are characterized by deadlines. When a missed deadline results in inconvenience or a diminished customer experience, we refer to this as soft real time. In contrast, hardreal-timesystemsareconsideredfailedwhenadeadlineismissed. • Kernel preemption wasthe firstsignificantfeatureintheLinuxkernelthataddressedsystemwidelatency. • Recent Linux kernels support several preemption modes, ranging from no preemption to full real-timepreemption. • The real-time patch adds several key features to the Linux kernel, resulting in reliable low latencies. • The real-time patch includes several important measurement tools to aid in debugging and characterizingareal-timeLinuximplementation. 17.5.1.SuggestionsforAdditionalReading LinuxKernelDevelopment,2ndEdition RobertLove NovellPress,2005 AppendixA.GNUPublicLicense ThisisanexactreproductionoftheGLPlicenseasauthoredandpublishedbytheFreeSoftware Foundation.Anelectroniccopycanbeobtainedatwww.fsf.org. Version2,June1991 Copyright©1989,1991FreeSoftwareFoundation,Inc. 51FranklinStreet,FifthFloor,Boston,MA02110-1301,USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changingitisnotallowed. Preamble Thelicensesformostsoftwarearedesignedtotakeawayyourfreedomtoshareandchangeit.By contrast,theGNUGeneralPublicLicenseisintendedtoguaranteeyourfreedomtoshareandchange freesoftwaretomakesurethesoftwareisfreeforallitsusers.ThisGeneralPublicLicenseapplies tomostoftheFreeSoftwareFoundation'ssoftwareandtoanyotherprogramwhoseauthorscommit tousingit.(SomeotherFreeSoftwareFoundationsoftwareiscoveredbytheGNULesserGeneral PublicLicenseinstead.)Youcanapplyittoyourprograms,too. When we speak of free software, we are referring to freedom, not price. Our General Public Licensesaredesignedtomakesurethatyouhavethefreedomtodistributecopiesoffreesoftware (andchargeforthisserviceifyouwish),thatyoureceivesourcecodeorcangetitifyouwantit,that youcanchangethesoftwareorusepiecesofitinnewfreeprograms;andthatyouknowyoucando thesethings. Toprotectyourrights,weneedtomakerestrictionsthatforbidanyonetodenyyoutheserightsor toaskyoutosurrendertherights.Theserestrictionstranslatetocertainresponsibilitiesforyouifyou distributecopiesofthesoftware,orifyoumodifyit. Forexample,ifyoudistributecopiesofsuchaprogram,whethergratisorforafee,youmustgive therecipientsalltherightsthatyouhave.Youmustmakesurethatthey,too,receiveorcangetthe sourcecode.Andyoumustshowthemthesetermssotheyknowtheirrights. Weprotectyourrightswithtwosteps:(1)copyrightthesoftware,and(2)offeryouthislicense whichgivesyoulegalpermissiontocopy,distributeand/ormodifythesoftware. Also,foreachauthor'sprotectionandours,wewanttomakecertainthateveryoneunderstands that there is no warranty for this free software. If the software is modified by someone else and passedon,wewantitsrecipientstoknowthatwhattheyhaveisnottheoriginal,sothatanyproblems introducedbyotherswillnotreflectontheoriginalauthors'reputations. Finally, any free program is threatened constantly by software patents. We wish to avoid the dangerthatredistributorsofafreeprogramwillindividuallyobtainpatentlicenses,ineffectmaking theprogramproprietary.Topreventthis,wehavemadeitclearthatanypatentmustbelicensedfor everyone'sfreeuseornotlicensedatall. Theprecisetermsandconditionsforcopying,distributionandmodificationfollow. TermsandConditionsforCopying,Distributionand Modification 1. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program",below,referstoanysuchprogramorwork,anda"workbasedontheProgram"means eithertheProgramoranyderivativeworkundercopyrightlaw:thatistosay,aworkcontainingthe Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licenseeisaddressedas"you". Activitiesotherthancopying,distributionandmodificationarenotcoveredbythisLicense;they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of havingbeenmadebyrunningtheProgram).WhetherthatistruedependsonwhattheProgramdoes. 2.YoumaycopyanddistributeverbatimcopiesoftheProgram'ssourcecodeasyoureceiveit,in anymedium,providedthatyouconspicuouslyandappropriatelypublishoneachcopyanappropriate copyrightnoticeanddisclaimerofwarranty;keepintactallthenoticesthatrefertothisLicenseand totheabsenceofanywarranty;andgiveanyotherrecipientsoftheProgramacopyofthisLicense alongwiththeProgram. Youmaychargeafeeforthephysicalactoftransferringacopy,andyoumayatyouroptionoffer warrantyprotectioninexchangeforafee. 3.YoumaymodifyyourcopyorcopiesoftheProgramoranyportionofit,thusformingawork basedontheProgram,andcopyanddistributesuchmodificationsorworkunderthetermsofSection 1above,providedthatyoualsomeetalloftheseconditions: a.Youmustcausethemodifiedfilestocarryprominentnoticesstatingthatyouchangedthefiles andthedateofanychange. b.Youmustcauseanyworkthatyoudistributeorpublish,thatinwholeorinpartcontainsoris derived from the Program or any part thereof, to be licensed as a whole at no charge to all third partiesunderthetermsofthisLicense. c.Ifthemodifiedprogramnormallyreadscommandsinteractivelywhenrun,youmustcauseit, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions,andtellingtheuserhowtoviewacopyofthisLicense.(Exception:iftheProgramitself isinteractivebutdoesnotnormallyprintsuchanannouncement,yourworkbasedontheProgramis notrequiredtoprintanannouncement.) Theserequirementsapplytothemodifiedworkasawhole.Ifidentifiablesectionsofthatwork arenotderivedfromtheProgram,andcanbereasonablyconsideredindependentandseparateworks in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a workbasedontheProgram,thedistributionofthewholemustbeonthetermsofthisLicense,whose permissionsforotherlicenseesextendtotheentirewhole,andthustoeachandeverypartregardless ofwhowroteit. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely byyou; rather,theintentisto exercisetheright to controlthedistributionofderivative or collectiveworksbasedontheProgram. In addition, mere aggregation of another work not based on the Program with the Program (or withaworkbasedontheProgram)onavolumeofastorageordistributionmediumdoesnotbring theotherworkunderthescopeofthisLicense. 4.YoumaycopyanddistributetheProgram(oraworkbasedonit,underSection2)inobject codeorexecutableformunderthetermsofSections1and2aboveprovidedthatyoualsodooneof thefollowing: a.Accompanyitwiththecompletecorrespondingmachine-readablesourcecode,whichmustbe distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange;or, b.Accompanyitwithawrittenoffer,validforatleastthreeyears,togiveanythirdparty,fora charge no more than your cost of physically performing source distribution, a complete machinereadablecopyofthecorrespondingsourcecode,tobedistributedunderthetermsofSections1and2 aboveonamediumcustomarilyusedforsoftwareinterchange;or, c. Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you receivedtheprograminobjectcodeorexecutableformwithsuchanoffer,inaccordwithSubsection babove.) Thesourcecodeforaworkmeansthepreferredformoftheworkformakingmodificationstoit. Foranexecutablework,completesourcecodemeansallthesourcecodeforallmodulesitcontains, plus any associated interface definition files, plus the scripts used to control compilation and installationoftheexecutable.However,asaspecialexception,thesourcecodedistributedneednot include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unlessthatcomponentitselfaccompaniestheexecutable. Ifdistributionofexecutableorobjectcodeismadebyofferingaccesstocopyfromadesignated place, then offering equivalent access to copy the source code from the same place counts as distributionofthesourcecode,eventhoughthirdpartiesarenotcompelledtocopythesourcealong withtheobjectcode. 5.Youmaynotcopy,modify,sublicense,ordistributetheProgramexceptasexpresslyprovided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have receivedcopies,orrights,fromyouunderthisLicensewillnothavetheirlicensesterminatedsolong assuchpartiesremaininfullcompliance. 6. You are not required to accept this License, since you have not signed it. However, nothing elsegrantsyoupermissiontomodifyordistributetheProgramoritsderivativeworks.Theseactions areprohibitedbylawifyoudonotacceptthisLicense.Therefore,bymodifyingordistributingthe Program(oranyworkbasedontheProgram),youindicateyouracceptanceofthisLicensetodoso, andallitstermsandconditionsforcopying,distributingormodifyingtheProgramorworksbasedon it. 7. Each time you redistribute the Program (or any work based on the Program), the recipient automaticallyreceivesalicensefromtheoriginallicensortocopy,distributeormodifytheProgram subjecttothesetermsandconditions.Youmaynotimposeanyfurtherrestrictionsontherecipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third partiestothisLicense. 8.If,asaconsequenceofacourtjudgmentorallegationofpatentinfringementorforanyother reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreementorotherwise)thatcontradicttheconditionsofthisLicense,theydonotexcuseyoufrom theconditionsofthisLicense.Ifyoucannotdistributesoastosatisfysimultaneouslyyourobligations underthisLicenseandanyotherpertinentobligations,thenasaconsequenceyoumaynotdistribute theProgramatall.Forexample,ifapatentlicensewouldnotpermitroyalty-freeredistributionofthe Programby allthose who receivecopiesdirectly orindirectlythroughyou,then theonlywayyou couldsatisfybothitandthisLicensewouldbetorefrainentirelyfromdistributionoftheProgram. Ifanyportionofthissectionisheldinvalidorunenforceableunderanyparticularcircumstance, thebalanceofthesectionisintendedtoapplyandthesectionasawholeisintendedtoapplyinother circumstances. Itisnotthepurposeofthissectiontoinduceyoutoinfringeanypatentsorotherpropertyright claimsortocontestvalidityofanysuchclaims;thissectionhasthesolepurposeofprotectingthe integrityofthefreesoftwaredistributionsystem,whichisimplementedbypubliclicensepractices. Manypeoplehavemadegenerouscontributionstothewiderangeofsoftwaredistributedthroughthat systeminrelianceonconsistentapplicationofthatsystem;itisuptotheauthor/donortodecideifhe orsheiswillingtodistributesoftwarethroughanyothersystemandalicenseecannotimposethat choice. Thissectionisintendedtomakethoroughlyclearwhatisbelievedtobeaconsequenceoftherest ofthisLicense. 9.Ifthedistributionand/oruseoftheProgramisrestrictedincertaincountrieseitherbypatents orbycopyrightedinterfaces,theoriginalcopyrightholderwhoplacestheProgramunderthisLicense mayaddanexplicitgeographicaldistributionlimitationexcludingthosecountries,sothatdistribution ispermittedonlyinoramongcountriesnotthusexcluded.Insuchcase,thisLicenseincorporatesthe limitationasifwritteninthebodyofthisLicense. 10. The Free Software Foundation may publish revised and/or new versions of the General PublicLicensefromtimetotime.Suchnewversionswillbesimilarinspirittothepresentversion, butmaydifferindetailtoaddressnewproblemsorconcerns. Eachversionisgivenadistinguishingversionnumber.IftheProgramspecifiesaversionnumber ofthisLicensewhichappliestoitand"anylaterversion",youhavetheoptionoffollowingtheterms and conditions either of that version or of any later version published by the Free Software Foundation.IftheProgramdoesnotspecifyaversionnumberofthisLicense,youmaychooseany versioneverpublishedbytheFreeSoftwareFoundation. 11.IfyouwishtoincorporatepartsoftheProgramintootherfreeprogramswhosedistribution conditionsaredifferent,writetotheauthortoaskforpermission.Forsoftwarewhichiscopyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptionsforthis.Ourdecisionwillbeguidedbythetwogoalsofpreservingthefreestatusofall derivativesofourfreesoftwareandofpromotingthesharingandreuseofsoftwaregenerally. NoWarranty 12. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHERPARTIESPROVIDETHEPROGRAM"ASIS"WITHOUTWARRANTYOFANYKIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIESOFMERCHANTABILITYANDFITNESSFORAPARTICULARPURPOSE.THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARYSERVICING,REPAIRORCORRECTION. 13. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES AppendixB.U-BootConfigurableCommands U-Boothasmorethan60configurablecommands.ThesearesummarizedhereinTableB-1from arecentU-Bootsnapshot.Inadditiontothesearealargenumberofnonstandardcommands,someof which depend on specific hardware or are experimental. For the complete and up-to-date listing, consultthesourcecode.Thecommandsaredefinedinthe.../include/cmd_confdefs.hheaderfilefrom thetop-levelU-Bootsourcedirectory. TableB-1.U-BootConfigurableCommands CommandSet Commands CFG_CMD_BDI bdinfo CFG_CMD_LOADS loads CFG_CMD_LOADB loadb CFG_CMD_IMI iminfo CFG_CMD_CACHE icache,dcache CFG_CMD_FLASH flinfo,erase,protect CFG_CMD_MEMORY md,mm,nm,mw,cp,cmp,crc,base,loop,mtest CFG_CMD_NET bootp,tftpboot,rarpboot CFG_CMD_ENV saveenv CFG_CMD_KGDB kgdb CFG_CMD_PCMCIA PCMCIAsupport CFG_CMD_IDE IDEharddisksupport CFG_CMD_PCI pciinfo CFG_CMD_IRQ irqinfo CFG_CMD_BOOTD bootd CFG_CMD_CONSOLE coninfo CFG_CMD_EEPROM EEPROMread/writesupport CFG_CMD_ASKENV askforenvironmentvariable CFG_CMD_RUN runcommandinenvironmentvariable CFG_CMD_ECHO echoarguments CFG_CMD_I2C I2Cserialbussupport CFG_CMD_REGINFO Registerdump CFG_CMD_IMMAP IMMRdumpsupport CFG_CMD_DATE SupportforRTC,date/time,andsoon. CFG_CMD_DHCP DHCPsupport CFG_CMD_BEDBUG IncludesBedBugdebugger CFG_CMD_FDC Floppydisksupport CFG_CMD_SCSI SCSIsupport CFG_CMD_AUTOSCRIPT Autoscriptsupport CFG_CMD_MII CFG_CMD_SETGETDCR CFG_CMD_BSP CFG_CMD_ELF CFG_CMD_MISC CFG_CMD_USB CFG_CMD_DOC CFG_CMD_JFFS2 CFG_CMD_DTT CFG_CMD_SDRAM CFG_CMD_DIAG CFG_CMD_FPGA CFG_CMD_HWFLOW CFG_CMD_SAVES CFG_CMD_SPI CFG_CMD_FDOS CFG_CMD_VFD CFG_CMD_NAND CFG_CMD_BMP CFG_CMD_PORTIO CFG_CMD_PING CFG_CMD_MMC CFG_CMD_FAT CFG_CMD_IMLS CFG_CMD_ITEST CFG_CMD_NFS CFG_CMD_REISER CFG_CMD_CDP CFG_CMD_XIMG CFG_CMD_UNIVERSE CFG_CMD_EXT2 CFG_CMD_SNTP CFG_CMD_DISPLAY MIIsupport DCRsupporton4xx Board-specificfunctions ELF(VxWorks)load/bootcommand Miscellaneousfunctions,suchassleep USBsupport Disk-on-chipsupport JFFS2support Digitalthermandthermostat SDRAMDIMMSPDinfoprintout Diagnostics FPGAconfigurationsupport RTS/CTShardwareflowcontrol SavesSrecorddump SPIutility FloppyDOSsupport VFDsupport(TRAB) NANDsupport BMPsupport PortI/O Pingsupport MMCsupport FATsupport Listsallfoundimages Integer(andstring)test NFSsupport Reiserfssupport CiscoDiscoveryProtocol Loadspartofmulti-image TundraUniversesupport EXT2support SNTPsupport Displaysupport AppendixC.BusyBoxCommands BusyBox has many useful commands. Here is a list of the commands documented in a recent BusyBoxsnapshot. addgroup Addsagrouptothesystem adduser Addsausertothesystem adjtimex Readsandoptionallysetssystemtimebaseparameters ar Extractsorlistsfilesfromanararchive arping PingshostsbyARPrequests/replies ash Theashshell(commandinterpreter) awk Pattern-scanningandprocessinglanguage basename Stripsdirectorypathandsuffixesfromfiles bunzip2 Uncompressesafile(orstandardinputifnoinputfilespecified) bzcat Uncompressestostdout cal Displaysacalendar cat Concatenatesfile(s)andprintsthemtostdout chgrp Changesthegroupmembershipofeachfile chmod Changesfileaccesspermissions chown Changestheownerand/orgroupoffile(s) chroot Runsthecommandwithrootdirectorysettonewroot chvt Changestheforegroundvirtualterminalto/dev/ttyN clear Clearsscreen cmp Comparesfiles cp Copiesfiles cpio Extractsorlistsfilesfromacpioarchive crond BusyBox'sversionofcrondaemon crontab Managescrontabcontrolfile cut Printsselectedfieldsfromeachinputfiletostandardoutput date Displaysorsetsthesystemtime dc TinyRPNcalculator dd Copiesafile,convertingandformattingaccordingtooptions deallocvt Deallocatesunusedvirtualterminal/dev/ttyN delgroup Deletesagroupfromthesystem deluser Deletesauserfromthesystem devfsd Obsoletedaemonformanagingdevfspermissionsandolddevicenamesymlinks df Printsthefilesystemspaceusedandspaceavailable dirname Stripsanondirectorysuffixfromafilename dmesg dos2unix dpkg dpkg-deb du dumpkmap dumpleases echo env expr false fbset fdflush fdformat fdisk find fold free freeramdisk fsckminix ftpget ftpput getopt getty grep gunzip gzip halt hdparm head hexdump hostid hostname httpd hwclock id ifconfig ifdown Printsorcontrolsthekernelringbuffer ConvertsafilefromDOSformattoUNIXformat Utilitytoinstall,remove,andmanageDebianpackages PerformsactionsonDebianpackages(debs) Summarizesdiskspaceusedforeachfileand/ordirectory Printsabinarykeyboard-translationtabletostandardoutput DisplaystheDHCPleasesgrantedbyudhcpd PrintsthespecifiedARGstostdout Printsthecurrentenvironmentorrunsaprogramaftersetting Printsthevalueofanexpressiontostandardoutput ReturnsanexitcodeofFALSE(1) Showsandmodifiesframebuffersettings Forcesfloppydiskdrivetodetectdiskchange Low-level-formatsafloppydisk Changespartitiontable Searchesforfilesinadirectoryhierarchy Wrapsinputlinesineachfile Displaystheamountoffreeandusedsystemmemory Freesallmemoryusedbythespecifiedramdisk PerformsaconsistencycheckforMINIXfilesystems RetrievesaremotefileviaFTP StoresalocalfileonaremotemachineviaFTP Parsescommandoptions Opensatty,promptsforaloginname,andtheninvokes/bin/login SearchesforPATTERNineachfileorstandardinput Uncompressesfile(orstandardinput) Compressesfile(s)withmaximumcompression Haltsthesystem Gets/setsharddiskparameters Printsfirst10linesofeachfiletostandardoutput Dumpsfilesinuser-specifiedbinary,octal,hex,character,ordecimalformat Printsaunique32-bitidentifierforthemachine Getsorsetsthehostname Listensforincominghttpserverrequests Queriesandsetsthehardwareclock(RTC) PrintsinformationforUSERNAMEorthecurrentuser Configuresanetworkinterface Deconfiguresaninterface ifup inetd init insmod install ip ipaddr ipcalc iplink iproute iptunnel kill killall klogd lash last length ln loadfont loadkmap logger login logname logread losetup ls lsmod makedevs md5sum mesg mkdir mkfifo mkfsminix mknod mkswap mktemp modprobe more mount Configureaninterface Listenssfornetworkconnectionsandlaunchesprograms BusyBoxversionofinit Loadsthespecifiedkernelmodulesintothekernel Copiesfilesandsetsattributes TCP/IPconfigurationutility Manipulatesinterfaceaddresses CalculatesIPnetworksettingsfromanIPaddress Manipulatesinterfacesettings Displays/setsroutingtableentries BusyBoxiptunnelutility Sendsasignal(defaultisSIGTERM)tothespecifiedprocess(es) Sendsasignal(defaultisSIGTERM)tothespecifiedprocess(es) Kernellogger TheBusyBoxLAmeSHell(commandinterpreter) Showsalistingofthelastuserswhologgedintothesystem PrintsthelengthofthespecifiedSTRING CreatesalinknamedLINK_NAMEorDIRECTORYtothespecifiedTARGET Loadsaconsolefontfromstandardinput Loadsabinarykeyboard-translationtablefromstandardinput WritesMESSAGEtothesystemlog Beginsanewsessiononthesystem Printsthenameofthecurrentuser Showsthemessagesfromsyslogd AssociatesLOOPDEVICEwithfile Listsdirectorycontents Liststhecurrentlyloadedkernelmodules Createsarangeofblockorcharacterspecialfiles PrintsorchecksMD5checksums mesgcontrolswriteaccesstoyourterminal Createsdirectoryentries Createsanamedpipe(identicaltomknodnamep) MakesaMINIXfilesystem Createsaspecialfile(block,character,orpipe) Preparesadiskpartitiontobeusedasaswappartition CreatesatemporaryfilewithitsnamebasedonTEMPLATE Usedforhigh-levelmoduleloadingandunloading Filterforviewingfilesonescreenfulatatime Mountsafilesystem mt mv nameif nc netstat nslookup od openvt passwd patch pidof ping ping6 pivot_root poweroff printf ps pwd rdate readlink realpath reboot renice reset rm rmdir rmmod route rpm rpm2cpio run-parts rx sed seq setkeycodes sha1sum sleep sort Controlsmagnetictapedriveoperation Renamesand/ormovesfiles Renamesanetworkinterfacewhileinthedownstate NetcatopensapipetoIP:port NetstatdisplaysLinuxnetworkinginformation QueriesthenameserverfortheIPaddressofthegivenhost Dumpsfilesinoctalandotherformats Startsacommandonanewvirtualterminal Changesauserpassword BusyBoximplementationofpatch GetsPIDofnamedprocess SendsICMPECHO_REQUESTpacketstonetworkhosts SendsICMPECHO_REQUESTpacketstonetworkhosts Changestherootfilesystem Haltsthesystemandrequeststhatthekernelshutoffthepower Formatsandprintsargumentsaccordingtouserformat Reportsprocessstatus Printsthefullfilenameofthecurrentworkingdirectory GetsandpossiblysetsthesystemdateandtimefromaremoteHOST Displaysthevalueofasymboliclink Returnstheabsolutepathnamesofagivenargument Rebootsthesystem Changespriorityofrunningprocessesinallowedprioritiesrange Resetsthescreen Removes(unlink)file(s) Removesdirectory(ies),iftheyareempty Unloadsthespecifiedkernelmodulesfromthekernel Editsthekernel'sroutingtables ManipulatesRPMpackages Outputsacpioarchiveoftherpmfile Runsabunchofscriptsinadirectory Receivesafileusingthexmodemprotocol BusyboxStreamEditorimplementation Printsarangeofnumberstostandardoutput Setsentriesintothekernel'sscancode-to-keycodemap PrintsorchecksSHA1checksums Delayforspecifiedamountoftime Sortslinesoftextinthespecifiedfiles start-stop-daemon Programtostartandstopservices strings stty su sulogin swapoff swapon sync sysctl syslogd tail tar tee telnet telnetd test tftp time top touch tr traceroute true tty udhcpc udhcpd umount uname uncompress uniq unix2dos unzip uptime usleep uudecode uuencode vconfig vi Displaysprintablestringsinabinaryfile Displaysandmodifiesterminalsettings ChangesuserIDorbecomeroot Singleuserlogin Disablesvirtualmemorypageswapping Enablesvirtualmemorypageswapping Writesallbufferedfilesystemblockstodisk Configureskernelparametersatruntime Linuxsystemandkernel-loggingutility Printslast10linesofeachfiletostandardoutput Creates,extracts,orlistsfilesfromatarfile Copiesstandardinputtoeachfileandalsotostandardoutput BusyBoxTelnetclientimplementation BusyBoxTelnetserverimplementation Checksfiletypesandcomparesvalues,returninganexit TransfersafileusingTFTPprotocol Measurestimeusedbyaprogram Providesaviewofprocessoractivityinrealtime Updatesthelast-modifieddateonthegivenFILE[s] Translates,squeezes,and/ordeletescharacters TracestherouteIPpacketsfollow ReturnsanexitcodeoftrUE(0) Printsthefilenameoftheterminalconnectedtostandardinput BusyBoxDHCPclientimplementation BusyBoxDHCPserverimplementation Unmountfilesystems Printscertainsysteminformation UncompressesZfile(s) DiscardsallbutoneofsuccessiveidenticallinesfromINPUT ConvertsfilefromUNIXformattoDOSformat ExtractsfilesfromZIParchives Displaysthetimesincethelastboot Pausesfornmicroseconds Uudecodesafilethatisuuencoded Uuencodesafile LetsyoucreateandremovevirtualEthernetdevices BusyBoxvieditor vlock Locksavirtualterminalandrequiresapasswordtounlockit watch watchdog wc wget which who whoami xargs yes zcat Executesaprogramperiodically Periodicallywritestoaspecifiedwatchdogdevice Printsline,word,andbytecountsforeachfile RetrievesfilesviaHTTPorFTP Locatesacommandonthecurrentpath Printsthecurrentusernamesandrelatedinformation PrintstheusernameassociatedwiththecurrenteffectiveuserID Executesacommandoneveryitemgivenbystandardinput RepeatedlyoutputsalinewithallspecifiedSTRING(s),ory Uncompressestostdout AppendixD.SDRAMInterfaceConsiderations At first glance, programming an SDRAM controller can seem like a formidable task. Indeed, numerous Synchronous Dynamic Random Access Memory (DRAM) technologies have been developed. In a never-ending quest for performance and density, many different architectures and modesofoperationhavebeendeveloped. We examine the AMCC PowerPC 405GP processor for this discussion of SDRAM interface considerations.Youmightwanttohaveacopyoftheusermanualtoreferencewhileweexplorethe issuesrelatedtoSDRAMinterfacing.ThisdocumentisreferencedinSectionD.4.1,"Suggestionsfor AdditionalReading." D.1.SDRAMBasics TounderstandSDRAMsetup,itisnecessarytounderstandthebasicsofhowanSDRAMdevice operates.Withoutgoingintothedetailsofthehardwaredesign,anSDRAMdeviceisorganizedasa matrixofcells,withanumberofaddressbitsdedicatedtorowaddressingandanumberdedicatedto columnaddressing.FigureD-1illustratesthis. FigureD-1.SimplifiedSDRAMblockdiagram Inside the memory matrix, the circuitry is quite complex. A simplified example of a read operationisasfollows:Agivenmemorylocationisreferencedbyplacingarowaddressontherow addresslinesandthenplacingacolumnaddressonthecolumnaddresslines.Aftersometimehas passed,thedatastoredatthelocationaddressedbytherowandcolumninputsaremadeavailableto theprocessoronthedatabus. TheprocessoroutputsarowaddressontheSDRAMaddressbusandassertsitsRowAddress Select(RAS)signal.AfterashortpreprogrammeddelaytoallowtheSDRAMcircuitrytocapture the row address, the processor outputs a column address and asserts its Column Address Select (CAS) signal. The SDRAM controller translates the actual physical memory address into row and column addresses. Many SDRAM controllers can be configured with the row and column width sizes;thePPC405GPisoneofthoseexamples.Lateryouwillseethatthismustbeconfiguredaspart oftheSDRAMcontrollersetup. Thisexampleismuchsimplified,buttheconceptsarethesame.Aburstread,forexample,which readsfourmemorylocationsatonce,outputsasingleRASandCAScycle,andtheinternalSDRAM circuitryautomaticallyincrementsthecolumnaddressforthesubsequentthreelocationsoftheburst read, eliminating the need for the processor to issue four separate CAS cycles. This is but one exampleofperformanceoptimization.Thebestwaytounderstandthisistoabsorbthedetailsofan actual memory chip. An example of a well-written data sheet is included in Section D.4.1, "SuggestionsforAdditionalReading." D.1.1.SDRAMRefresh An SDRAM is composed of a single transistor and a capacitor. The transistor supplies the charge,andthecapacitor'sjobistoretain(store)thevalueoftheindividualcell.Forreasonsbeyond thescopeofthisdiscussion,thecapacitorcanholdthevalueforonlyasmallduration.Oneofthe fundamental concepts of dynamic memory is that the capacitors representing each cell must be periodicallyrechargedtomaintaintheirvalue.ThisisreferredtoasSDRAMrefresh. A refresh cycle is a special memory cycle that neither reads nor writes data to the memory. It simply performs the required refresh cycle. One of the primary responsibilities of an SDRAM controlleristoguaranteethatrefreshcyclesareissuedintimetomeetthechip'srequirements. The chip manufacturers specify minimum refresh intervals, and it is the designer's job to guaranteeit.UsuallytheSDRAMcontrollercanbeconfigureddirectlytoselecttherefreshinterval. The PowerPC 405GP presented here has a register specifically for this purpose. We will see this shortly. D.2.Clocking ThetermsynchronousimpliesthatthedatareadandwritecyclesofanSDRAMdevicecoincide withtheclocksignalfromtheCPU.SDRSDRAMisreadandwrittenoneachSDRAMclockcycle. DDRSDRAMisreadandwrittentwiceoneachclockcycle,onceontherisingedgeoftheclockand onceonthefallingedge. Modernprocessorshavecomplexclockingsubsystems.Manyhavemultipleclockratesthatare used for different parts of the system. A typical processor uses a relatively low-frequency crystalgenerated clock source for its primary clock signal. A phase locked loop internal to the processor generatestheCPU'sprimaryclock(theclockratewespeakofwhencomparingprocessorspeeds). Because the CPU typically runs much faster than the memory subsystem, the processor generates a submultiple of the main CPU clock to feed to the SDRAM subsystem. You need to configure this clockingratioforyourparticularCPUandSDRAMcombination. TheprocessorandmemorysubsystemclocksmustbecorrectlyconfiguredforyourSDRAMto workproperly.Yourprocessormanualcontainsasectiononclocksetupandmanagement,andyou mustconsultthistoproperlysetupyourparticularboarddesign. TheAMCC405GPistypicalofprocessorsofitsfeatureset.Ittakesasinglecrystal-generated clockinputsourceandgenerates severalinternalandexternalclocksrequiredofitssubsystems.It generatesclocksfortheCPU,PCIinterface,OnboardPeripheralBus(OPB),ProcessorLocalBus (PLB), Memory Clock (MemClk), and several internal clocks for peripherals such as timer and UARTblocks.AtypicalconfigurationmightlooklikethoseinTableD-1. TableD-1.TypicalPPC405GPClockConfiguration Clock Rate Comments Crystal 33MHz Fundamentalreferencesuppliedtoprocessor reference CPU Derivedfromprocessor'sinternalPLL,controlledbyhardwarepinstrappingand 133MHz clock registersettings. DerivedfromCPUclockandconfiguredviahardwarepinstrappingandregister PLB 66MHz settings.Usedforinternalprocessorlocalbusdatainterchangeamongitshighclock speedmodules. OPB DerivedfromPLBclockandconfiguredviaregistersettings.Usedforinternal 66MHz clock connectionofperipheralsthatdonotneedhigh-speedconnection. PCI 33MHz DerivedfromPLBclockandconfiguredviaregistersettings. clock DrivestheSDRAMchipsdirectly.DerivedfromCPUclockandconfiguredvia MemClk 100MHz registersettings. Decisions about clock setup normally must be made at hardware design time. Pin strapping options determine initial clock configurations upon application of power to the processor. Some control over derived clocks is often available by setting divider bits accessible through processor internalregistersdedicatedtoclockandsubsystemcontrol.Intheexamplewepresentherebasedon the405GP,finalclockconfigurationisdeterminedbypinstrappingandfirmwareconfiguration.Itis thebootloader'sresponsibilitytosettheinitialdividersandanyotherclockoptionsconfigurablevia processorregisterbitsveryearlyafterpowerisapplied. D.3.SDRAMSetup After the clocks have been configured, the next step is to configure the SDRAM controller. Controllersvarywidelyfromprocessortoprocessor,buttheendresultisalwaysthesame:Youmust provide the correct clocking and timing values to enable and optimize the performance of the SDRAMsubsystem. Aswithothermaterialinthisbook,thereisnosubstitutefordetailedknowledgeofthehardware youaretryingtoconfigure.ThisisespeciallysoforSDRAM.Itisbeyondthescopeofthisappendix to explore the design of SDRAM, but some basics must be understood. Many manufacturers' data sheets on SDRAM devices contain helpful technical descriptions. You are urged to familiarize yourself with the content of these data sheets. You don't need a degree in hardware engineering to understandwhatmustbedonetoproperlyconfigureyourSDRAMsubsystem,butyouneedtoinvest insomelevelofunderstanding. Here we examine how the SDRAM controller is configured on the 405GP processor as configuredbytheU-BootbootloaderwecoveredinChapter7,"Bootloaders."RecallfromChapter7 that U-Boot provides a hook for SDRAM initialization from the assembly language startup code found in start.S in the 4xx-specific cpu directory. Refer back to Section 7.4.4 "Board-Specific Initialization" in Chapter 7. Listing D-1 reproduces the sdram_init() function from U-Boot's .../cpu/ppc4xx/sdram.cfile. ListingD.ppc4xxsdram_init()fromU-Boot 01voidsdram_init(void) 02{ 03ulongsdtr1; 04ulongrtr; 05inti; 06 07/* 08*Supportfor100MHzand133MHzSDRAM 09*/ 10if(get_bus_freq(0)>100000000){ 11/* 12*133MHzSDRAM 13*/ 14sdtr1=0x01074015; 15rtr=0x07f00000; 16}else{ 17/* 18*default:100MHzSDRAM 19*/ 20sdtr1=0x0086400d; 21rtr=0x05f00000; 22} 23 24for(i=0;i<N_MB0CF;i++){ 25/* 26*Disablememorycontroller. 27*/ 28mtsdram0(mem_mcopt1,0x00000000); 29 30/* 31*SetMB0CFforbank0. 32*/ 33mtsdram0(mem_mb0cf,mb0cf[i].reg); 34mtsdram0(mem_sdtr1,sdtr1); 35mtsdram0(mem_rtr,rtr); 36 37udelay(200); 38 39/* 40*Setmemorycontrolleroptionsreg,MCOPT1. 41*SetDC_ENto'1'andBRD_PRFto'01'for16bytePLBBurst 42*read/prefetch. 43*/ 44mtsdram0(mem_mcopt1,0x80800000); 45 46udelay(10000); 47 48if(get_ram_size(0,mb0cf[i].size)==mb0cf[i].size){ 49/* 50*OK,sizedetected->alldone 51*/ 52return; 53} 54} 55} Thefirstactionreadsthepinstrappingonthe405GPprocessortodeterminethedesignvaluefor the SDRAM clock. In this case, we can see that two possible values are accommodated: 100MHz and133MHz.Basedonthischoice,constantsarechosenthatwillbeusedlaterinthefunctiontoset theappropriateregisterbitsintheSDRAMcontroller. Startingonline24,aloopisusedtosettheparametersforeachofuptofivepredefinedmemory sizes.Currently,U-Boothaslogictosupportasinglebankofmemorysizedat4MB,16MB,32MB, 64MB, or 128MB. These sizes are defined in a table called mb0cf in .../cpu/ppc4xx/sdram.c. The table associates a constant with each of these memory sizes, based on the value required in the 405GPmemorybankconfigurationregister.Theloopdoesthis: for(i=eachpossiblememorybanksize,largestfirst){ selecttimingconstantbasedonSDRAMclockspeed; disableSDRAMmemorycontroller; configurebank0withsize[i],timingconstants[i]; re-enableSDRAMmemorycontroller; runsimplememorytesttodynamicallydeterminesize; /*Thisisdoneusingget_ram_size()*/ if(testedsize==configuredsize)done; } ThissimplelogicsimplyplugsinthecorrecttimingconstantsintheSDRAMcontrollerbasedon SDRAMclockspeedandconfiguredmemorybanksizefromthehard-codedtableinU-Boot.Using this explanation, you can easily correlate the bank configuration values using the 405GP reference manual.Fora64MBDRAMsize,thememorybankcontrolregisterissetasfollows: MemoryBank0ControlRegister=0x000a4001 The PowerPC 405GP User's Manual describes the fields in Table D-2 for the memory bank 0 controlregister. TableD-2.405GPMemoryBank0-3ConfigurationRegisterFields Field Value Comments Bank Address 0x00 Startingmemoryaddressofthisbank. (BA) Size(SZ) 0x4 Sizeofthismemorybankinthiscase,64MB. Determinestheorganizationofmemory,includingthenumberofrowandcolumn Addressing bits.Inthiscase,Mode2=12rowaddressbits,andeither9or10column Mode 0x2 addressbits,anduptofourinternalSDRAMbanks.Thisdataisprovidedina (AM) tableinthe405GPuser'smanual. Bank Enablebitforthebankconfiguredbythisregister.Therearefourofthesememory Enable 0x1 bankconfigurationregistersinthe405GP. (BE) The values in this table must be determined by the designer, based on the choice of memory moduleinuseontheboard. Let'slookatatimingexampleformoredetailonthetimingrequirementsofatypicalSDRAM controller.Assuminga100MHzSDRAMclockspeedand64MBmemorysize,thetimingconstants selectedbythesdram_init()functioninListingD-1areselectedasfollows: SDRAMTimingRegister=0x0086400d RefreshTimingRegister=0x05f00000 ThePowerPC405GPUser'sManualdescribesthefieldsinTableD-3fortheSDRAMTiming Register. TableD-3.405GPSDRAMTimingRegisterFields Field Value Comments SDRAMCASLatency.ThisvaluecomesdirectlyfromtheSDRAMchip CAS specifications.Itisthedelayinclockcyclesrequiredbythechipbetween Latency 0x1 issuanceofthereadcommand(CASsignal)untilthedataisavailableonthedata (CASL) bus.Inthiscase,the0x1representstwoclockcycles,asseenfromthe405GP user'smanual. TheSDRAMPrechargecommanddeactivatesagivenrow.Incontrast,the Precharge Command Activatecommandenablesagivenrowforsubsequentaccess,suchasduringa toNext 0x1 burstcycle.ThistimingparameterenforcestheminimumtimebetweenPrecharge Activate toasubsequentActivatecycleandisdictatedbytheSDRAMchip.Thecorrect (PTA) valuemustbeobtainedfromtheSDRAMchipspecification.Inthiscase,0x1 representstwoclockcycles,asdeterminedfromthe405GPuser'smanual. Read/Write to ThistimingparameterenforcestheminimumtimedelaybetweenagivenSDRAM Precharge readorwritecommandtoasubsequentPrechargecommand.Thecorrectvalue 0x2 Command mustbeobtainedfromtheSDRAMchipspecification.Inthiscase,0x2represents Minimum threeclockcycles,asdeterminedfromthe405GPuser'smanual. (CTP) SDRAM Thistimingparameterenforcestheminimumtimedelaybetweenassertionof Command addressorcommandcycletobankselectcycle.Thecorrectvaluemustbe 0x1 Leadoff obtainedfromtheSDRAMchipspecification.Inthiscase,0x1representstwo (LDF) clockcycles,asdeterminedfromthe405GPuser'smanual. ThefinaltimingparameterconfiguredbytheU-BootexampleinListingD-1istherefreshtiming registervalue.Thisregisterrequiresasinglefieldthatdeterminestherefreshintervalenforcedbythe SDRAMcontroller.Thefieldrepresentingtheintervalistreatedasasimplecounterrunningatthe SDRAMclockfrequency.Intheexamplehere,weassumed100MHzastheSDRAMclockfrequency. Thevalueprogrammedintothisregisterinourexampleis0x05f0_0000.FromthePowerPC405GP User'sManual,wedeterminethatthiswillproducearefreshrequestevery15.2microseconds.As withtheothertimingparameters,thisvalueisdictatedbytheSDRAMchipspecifications. AtypicalSDRAMchiprequiresonerefreshcycleforeachrow.Eachrowmustberefreshedin theminimumtimespecifiedbythemanufacturer.InthechipreferencedinSectionD.4.1,"Suggestions for Additional Reading," the manufacturer specifies that 8,192 rows must be refreshed every 64 milliseconds. This requires generating a refresh cycle every 7.8 microseconds to meet the specificationsforthisparticulardevice. D.4.Summary SDRAMdevicesarequitecomplex.Thisappendixpresentedaverysimpleexampletohelpyou navigate the complexities of SDRAM controller setup. The SDRAM controllers perform a critical function and must be properly set up. There is no substitute to diving into a specification and digesting the information presented. The two example documents referenced in this appendix are excellentstartingpoints. D.4.1.SuggestionsforAdditionalReading AMCC405GPEmbeddedProcessorUser'sManual AMCCCorporation www.amcc.com/Embedded/ MicronTechnology,Inc. SynchronousDRAMMT48LC64M4A2DataSheet http://download.micron.com/pdf/datasheets/dram/sdram/256MSDRAM.pdf AppendixE.OpenSourceResources SourceRepositoriesandDeveloperInformation SeverallocationsontheWebfocusonLinuxdevelopment.Hereisalistofthemostimportant websitesforthevariousarchitecturesandprojects: Primarykernelsourcetree www.kernel.org PrimarykernelGITrepository www.kernel.org/git PowerPC-relateddevelopmentandmailinglists http://ozlabs.org/ MIPS-relateddevelopments www.linux-mips.org ARM-relatedLinuxdevelopment www.arm.linux.org.uk Primaryhomeforahugecollectionofopen-sourceprojects http://sourceforge.net MailingLists Hundreds, if not thousands, of mailing lists cater to every aspect of Linux and open-source development. Here are a few to consider. Make sure you familiarize yourself with mailing list etiquettebeforepostingtotheselists. Mostoftheselistsmaintainarchivesthataresearchable.Thisisthefirstplacethatyoushould consult.Inagreatmajorityofthecases,yourquestionhasalreadybeenaskedandanswered.Start yourreadinghere,foradviceonhowtobestusethepublicmaillists: TheLinuxKernelMailingListFAQ www.tux.org/lkml ListserverservingvariousLinuxkernel-relatedmaillists http://vger.kernel.org LinuxKernelMailingveryhighvolume,kerneldevelopmentonly http://vger.kernel.org/vger-lists.html#linux-kernel LinuxNewsandDevelopments Manynewssitesareworthbrowsingoccasionally.Someofthemorepopulararelistedhere. LinuxDevices.com www.linuxdevices.com PowerPCNewsandotherinformation http://penguinppc.org GeneralLinuxNewsandDevelopments www.lwn.net OpenSourceInsightandDiscussion Thefollowingpublicwebsitecontainsusefulinformationandeducationfocusingonlegalissues aroundopensource. www.open-bar.org AppendixF.SampleBDI-2000ConfigurationFile ;bdiGDBconfigurationfilefortheUEIPPC5200Board ;Revision1.0 ;Revision1.1(Addedserialportsetup) ;----------------------------------------------------------;4MBFlash(Am29DL323) ;128MBMicronDDRDRAM ; [INIT] ;initcoreregister WREGMSR0x00003002;MSR:FP,ME,RI WM320x800000000x00008000;MBAR:internalregistersat0x80000000 ;DefaultafterRESET,MBARsitsat0x80000000 ;becauseit'sPORvalueis0x0000_8000(!) WSPR3110x80000000;MBAR:saveinternalregisteroffset ;SPR311istheMBARinG2_LE WSPR2790x80000000;SPRG7:saveinternalmemoryoffsetReg:279 ;InitCDM(ClockDistributionModule) ;HardwareResetconfig{ ;ppc_pll_cfg[0..4]=01000b :XLB:Core->1:3 :Core:f(VCO)->1:2 :XLB:f(VCO)->1:6 ; ;xlb_clk_sel=0->XLB_CLK=f(sys)/4=132MHz ; ;sys_pll_cfg_1=0->NOP ;sys_pll_cfg_0=0->f(sys)=16xSYS_XTAL_IN=528MHz ;} ; ;CDMConfigurationRegister WM320x8000020c0x01000101 ;enableDDRMode ;ipb_clk_sel=1->XLB_CLK/2(ipb_clk=66MHz) ;pci_clk_sel=01->IPB_CLK/2 ;CS0Flash WM320x800000040x0000ff00;CS0start=0xff000000-Flashmemoryison CS0 WM320x800000080x0000ffff;CS0stop=0xffffffff ;IPBIRegisterandWaitStateEnable WM320x800000540x00050001;CSE:enableCS0,disableCSBOOT, ;Waitstateenable\ ;CS2alsoenabled WM320x800003000x00045d30;BOOTctrl ;bits0-7:WaitP(try0xff) ;bits8-15:WaitX(try0xff) ;bit16:Multiplexornon-mux'ed(0x0=non-muxed) ;bit17:reserved(Resetvalue=0x1,keepit) ;bit18:AckActive(0x0) ;bit19:CE(Enable)0x1 ;bits20-21:AddressSize(0x11=25/6bits) ;bits22:23:Datasizefield(0x01=16-bits) ;bits24:25:Bankbits(0x00) ;bits26-27:WaitType(0x11) ;bits28:WriteSwap(0x0=noswap) ;bits29:ReadSwap(0x0=noswap) ;bit30:WriteOnly(0x0=readenable) ;bit31:ReadOnly(0x0=writeenable) ;CS2LogicRegisters WM320x800000140x0000e00e WM320x800000180x0000efff ;LEDS: ;LED1-bits0-7 ;LED2-bits8-15 ;LED3-bits16-23 ;LED4-bits24-31 ;off=0x01 ;on=0x02 ;mm0xe00e20300x020202021(allon) ;mm0xe00e20300x010201021(2on,2off) WM320x800003080x00045b30;CS2ConfigurationRegister ;bits0-7:WaitP(try0xff) ;bits8-15:WaitX(try0xff) ;bit16:Multiplexornon-mux'ed(0x0= non-muxed) ;bit17:reserved(Resetvalue=0x1,keepit) ;bit18:AckActive(0x0) ;bit19:CE(Enable)0x1 ;bits20-21:AddressSize(0x10=24bits) ;bits22:23:Datasizefield(0x11=32-bits) ;bits24:25:Bankbits(0x00) ;bits26-27:WaitType(0x11) ;bits28:WriteSwap(0x0=noswap) ;bits29:ReadSwap(0x0=noswap) ;bit30:WriteOnly(0x0=readenable) ;bit31:ReadOnly(0x0=writeenable) WM320x800003180x01000000;MasterLPCEnable ; ;initSDRAMcontroller ; ;FortheUEIPPC5200Board, ;Micron46V32M16-75E(8MEGx16x4banks) ;64MBperChip,foratotalof128MB ;arrangedasasingle"space"(i.e1CS) ;withthefollowingconfiguration: ;8Mbx16x4banks ;Refreshcount8K ;Rowaddressing:8K(A0..12)13bits ;Columnaddressing:1K(A0..9)10bits ;BankAddressing:4(BA0..1)2bits ;KeyTimingParameters:(-75E) ;Clockrate(CL=2)133MHz ;DOWindow2.5ns ;AccessWindow:+/-75ns ;DQS-DQSkew:+0.5ns ;t(REFI):7.8usMAX ; ;InitializationRequirements(GeneralNotes) ;ThememoryMode/ExtendedModeregistersmustbe ;initializedduringthesystembootsequence.Butbefore ;writingtothecontrollerModeregister,themode_enand ;ckebitsintheControlregistermustbesetto1.After ;memoryinitializationiscomplete,theControlregister ;mode_enbitshouldbeclearedtopreventsubsequentaccess ;tothecontrollerModeregister. ;SDRAMinitsequence ;1)Setupandenablechipselects ;2)Setupconfigregisters ;3)SetupTAPDelay ;SetupandenableSDRAMCS WM320x800000340x0000001a;SDRAMCS0,128MB@0x00000000 WM320x800000380x08000000;SDRAMCS1,disabled@0x08000000 WM320x800001080x73722930;SDRAMConfig1Samsung ;AssumeCL=2 ;bits0-3:srd2rwp:inclocks(0x6) ;bits507:swt2rwp:inclocks->Datasheetsuggests ;0x3forDDR(0x3) ;bits8-11:rd_latency->forDDR0x7 ;bits13-15:act2rw->0x2 ;bit16:reserved ;bits17-19:pre2act->0x02 ;bits20-23:ref2act->0x09 ;bits25-27:wr_latency->forDDR0x03 ;bits28-31:Reserved WM320x8000010c0x46770000;SDRAMConfig2Samsung ;bits0-3:brd2rp->forDDR0x4 ;bits4-7:bwt2rwp->forDDR0x6 ;bits8-11:brd2wt->0x6 ;bits12-15:burst_length->0x07(bl-1) ;bits16-13:Reserved ;SetupinitialTapdelay WM320x800002040x18000000;Startintheendoftherange(24=0x18) Samsung WM320x800001040xf10f0f00;SDRAMControl(was0xd14f0000) ;bit0:mode_en(1=write) ;bit1:cke(MEM_CLK_EN) ;bit2:ddr(DDRmodeon) ;bit3:ref_en(Refreshenable) ;bits4-6:Reserved ;bit7:hi_addr(XLA[4:7]asrow/col ;mustbesetto'1''cuzweneed13RAbits ;fortheMicronchipabove ;bit8:reserved ;bit9:drive_rule-0x0 ;bit10-15:ref_interval,seeUM0x0f ;bits16-19:reserved ;bits20-23:dgs_oe[3:0](notsure) ;butIthinkthisisreq'dforDDR0xf ;bits24-28:Resv'd ;bit29:1=softrefresh ;bit301=soft_precharge ;bit31:reserved WM320x800001040xf10f0f02;SDRAMControl:prechargeall WM320x800001040xf10f0f04;SDRAMControl:refresh WM320x800001040xf10f0f04;SDRAMControl:refresh WM320x800001000x018d0000;SDRAMModeSamsung ;bits0-1:MEM_MBA-selectsstdorextendedMODEreg0x0 ;bits2-13:MEM_MA(seeDDRDRAMDatasheet) ;bits2-7:OperatingMode->0x0=normal ;bits8-10:CASLatency(CL)->SettoCL=2forDDR(0x2) ;bit11:BurstType:SequentialforPMC5200->0x0 ;bits12-14:Setto8forMPC5200->0x3 ;bit15:cmd=1forMODEREGWRITE WM320x800001040x710f0f00;SDRAMControl:LockModeRegister(was0x514f0000) ;***********Initializetheserialport*********** ;PinConfiguration WM320x80000b000x00008004;UART1 ;ResetPSC WM80x800020080X10;Reset-SelectMR1 WM160x800020040;ClockSelectRegister-0enablesbothRx& TxClocks WM320x800020400;SICR-UARTMode WM80x800020000x13;WriteMR1(defaultafterreset) ;8-bit,noparity WM80x800020000x07;WriteMR2(afterMR1)(onestopbit) WM80x800020180x0;Counter/TimerUpperReg(115.2KB) WM80x8000201c0x12;Counter/TimerLowerReg(divider=18) ;ResetandenableserialportRx/Tx WM80x800020080x20 WM80x800020080x30 WM80x800020080x05 ; ;definemaximaltransfersize TSZ40x800000000x80003FFF;internalregisters ; ;definethevalidmemorymap MMAP0x000000000x07FFFFFF;MemoryrangeforSDRAM MMAP0xFF0000000xFFFFFFFF;ROMspace MMAP0xE00E00000xE00EFFFF;PowerPCLogic MMAP0x800000000x8fffffff;DefaultMBAR MMAP0xC00000000XCFFFFFFF;LinuxKernal [TARGET] CPUTYPE5200;theCPUtype JTAGCLOCK0;use16MHzJTAGclock WORKSPACE0x80008000;workspaceforfastdownload WAKEUP1000;giveresettimetocomplete STARTUPRESET MEMDELAY2000;additionalmemoryaccessdelay BOOTADDR0xfff00100 REGLISTALL BREAKMODESOFT;orHARD POWERUP1000 WAKEUP500 MMUXLAT PTBASE0x000000f0 [HOST] IP192.168.1.9 FORMATELF LOADMANUAL;loadcodeMANUALorAUTOafterreset PROMPTuei> [FLASH] CHIPTYPE AM29BX16 ;Flash type (AM29F | AM29BX8 | AM29BX16 | I28BX8 | I28BX16) CHIPSIZE0x00400000;Thesizeofoneflashchipinbytes BUSWIDTH16;Thewidthoftheflashmemorybusinbits(8|16|32) WORKSPACE0x80008000;workspaceininternalSRAM FILEu-boot.bin FORMATBIN0xFFF00000 ERASE0xFFF00000;eraseasectorofflash ERASE0xFFF10000;eraseasectorofflash ERASE0xFFF20000;eraseasectorofflash ERASE0xFFF30000;eraseasectorofflash ERASE0xFFF40000;eraseasectorofflash [REGS] FILE$reg5200.def Примечания 1 SeeAppendixA,"GNUPublicLicense,"forthecompletetextofthelicense. n_1 2 Most professional development managers agree: You can download Linux without charge, but thereisacost(oftenasubstantialone)fordevelopmentanddeploymentofanyOSonanembedded platform.SeeSection1.3.1,"FreeVersusFreedom,"foradiscussionofcostelements. n_2 3 Ifallthecopyrightholdersagreed,thesoftwarecould,intheory,bereleasedunderanewlicense, averyunlikelyscenarioindeed! n_3 4 ATCAplatformsareintroducedinChapter3,"ProcessorBasics." n_4 5 Seewww.compactflash.org. n_5 6 Remember, you can change a 1 to a 0 a byte at a time, but you must erase the entire block to changeanybitfroma0backtoa1. n_6 7 Directlyinthelogicalsense.Theactualcircuitrymaycontainbusbuffersorbridgedevices,etc. n_7 8 WediscussramdiskfilesystemsinmuchdetailinChapter9,"FileSystems." n_8 9 Real-time clock modules often contain small amounts of nonvolatile storage, and Serial EEPROMsareanothercommonchoicefornonvolatilestorageofsmallamountsofdata. n_9 10 In this discussion, the word task is used to denote any thread of execution, regardless of the mechanismusedtospawn,manage,orscheduleit. n_10 11 Manygoodbookscoverthedetailsofvirtualmemorysystems.SeeSection2.5.1,"Suggestions forAdditionalReading,"attheendofthischapter,forrecommendations. n_11 12 However,thereisseldomagoodreasontochangeit. n_12 13 Thetermthreadhereisusedinthegenericsensetoindicateanysequentialflowofinstructions. n_13 14 Infact,itwouldn'tevencompileorlink,muchlessrun. n_14 15 This package is important enough to warrant its own chapter. Chapter 11, "BusyBox," covers BusyBoxindetail. n_15 16 LinuxhassupportforsomebasicprocessorsthatdonotcontainMMUs,butthisisnotconsidered amainstreamuseofLinux. n_16 17 32-bitand64-bitrefertothenativewidthoftheprocessor'smainfacilities,suchasitsexecution units,registerfileandaddressbus. n_17 18 32-bitand64-bitrefertothenativewidthoftheprocessor'smainfacilities,suchasitsexecution units,registerfileandaddressbus. n_18 19 Thesearetheauthor'sownopinionsbasedonmarketobservationandnotbasedonanyscientific data. n_19 20 On the Freescale website, navigate to Media Center, Press Releases. This one was dated 10/31/2005fromAustin,Texas. n_20 21 Source:www.mips.com/content/PressRoom/PressReleases/2003-12-22 n_21 22 Other peripherals include IrDA controller, LCD controller, 2 SPCs, Power management, DMA engine,RTC,Camerainterface,LCDcontroller,h/whardwareaccelerationofencryption/decryption, PCIhostcontroller,4SPCs,andSecurityengine. 23 Reported by ARM to be the top-selling toy during the Christmas 2005 shopping season in the UnitedStates. n_23 24 VMEbus isn't really a hardware reference platform, per se, but based on Eurocard physical standards,thelevelofcompatibilityamongmultiplevendorsqualifiesitforthelabel. n_24 25 Wetalkaboutthekernelbuildsystemandmakefilesshortly. n_25 26 Throughoutthisbook,threedotsprecedinganypathareusedtoindicatewhateverpathitmight takeonyoursystemtoreachthetop-levelLinuxsourcetree. n_26 27 ExecutableandLinkingFormat,ade-factostandardformatforbinaryexecutablefiles. n_27 28 Thelinkerscriptfilehasapeculiarsyntax.Thedetailscanbefoundinthedocumentationforthe GNUlinker. n_28 29 Not all these makefiles are directly involved in building the kernel. Some, for example, build documentationfiles. n_29 30 Incrementallinkingisatechniqueusedtogenerateanobjectmodulethatisintendedtobelinked againintoanotherobject.Inthisway,unresolvedsymbolsthatremainafterincrementallinkingdonot generateerrorstheyareresolvedatthenextlinkstage. n_30 31 Asmentioned,youcanusetheconfigurationeditorofyourchoice,suchasmakexconfigormake menuconfig. n_31 32 Betteryet,makeabackupcopyofyour.configfile. n_32 33 We have intentionally removed many options under ARM system type and Intel IXP4 xx ImplementationOptionstofitthepictureonthepage. n_33 34 Inactuality,thekernelbuildsystemisverycomplicated,butmostofthecomplexityiscleverly hidden from the average developer. As a result, it is relatively easy to add, modify, or delete configurationswithouthavingtobeanexpert. n_34 35 Always assume that features advance faster than the corresponding documentation, so treat the docsasaguideratherthanindisputablefacts. n_35 36 The kernel image is nearly always stored in compressed format, unless boot time is a critical issue. In this case, the image might be called uImage, a compressed vmlinux file with a U-Boot header.SeeChapter7,"Bootloaders." n_36 37 Thetermpiggywasoriginallyusedtodescribea"piggy-back"concept.Inthiscase,thebinary kernelimageispiggy-backedontothebootstraploadertoproducethecompositekernelimage. n_37 38 Not to be confused with the bootloader, a bootstrap loader can be considered a second-stage loader,wherethebootloaderitselfcanbethoughtofasafirst-stageloader. n_38 39 Thetermmachineasusedherereferstoaspecifichardwarereferenceplatform. n_39 40 Often called Instruction Pointer, the register which holds the address of the next machine instructioninmemory. n_40 41 Modifyinghead.Sforyourcustomplatformishighlydiscouraged.Thereisalmostalwaysabetter way.SeeChapter16,"PortingLinux,"foradditionalinformation. n_41 42 Modifyinghead.Sforyourcustomplatformishighlydiscouraged.Thereisalmostalwaysabetter way.SeeChapter16,"PortingLinux,"foradditionalinformation. n_42 43 Normally,thecompilerwillcomplainifavariableisdefinedstaticandneverreferencedinthe compilationunit.Becausethesevariablesarenotexplicitlyreferenced,thewarningwouldbeemitted withoutthisdirective. n_43 44 Youmighthavetolowerthedefaultloglevelonyoursystemtoseethesedebugmessages.Thisis describedinmanyreferencesaboutLinuxsystemadministration.Inanycase,youshouldseethemin thekernellogfile. n_44 45 Userlandisanoften-usedtermforanyprogram,library,script,oranythingelseinuserspace. n_45 46 Especiallythesectionsonfunctionattributes,typeattributes,andvariableattributes. n_46 47 Oftenembeddedsystemsdonothaveuseraccountsotherthanasinglerootuser. 48 BusyBoxcommandsarecoveredinChapter11. n_48 49 DevicenodesareexplainedindetailinChapter8. n_49 50 In actuality, modern Linux kernels create a userspace-like environment earlier in the boot sequenceforspecializedactivities,whicharebeyondthescopeofthisbook. n_50 51 Whenbusyboxisinvokedviatheshsymboliclink,itspawnsashell.Wecoverthisindetailin Chapter11. n_51 52 Thisinittabisaniceexampleofasmall,purpose-builtembeddedsystem. n_52 53 Itjustsohappensthatonthisparticularboard,ourphysicalSDRAMstartsat256MB. n_53 54 Out of necessity (space), this is a very simplified description of the sequence of events. The actualmechanismissimilarinconcept,butseveralsignificantdetailsareomittedforclarity.Youare encouraged to consult the kernel source code for more details. See .../init/main.c and .../init/do_mounts*.c. n_54 55 do_basic_setupiscalledfrom.../init/main.candcallsdo_initcalls().Thiscausesdrivermodule initializationroutinestobecalled.ThiswasdescribedindetailinChapter5andshowninListing510. n_55 56 Someembeddeddesignsprotectthebootloaderandprovidecallbackstobootloaderroutines,but thisisalmostneveragooddesignapproach.Linuxisfarmorecapablethanbootloaders,sothereis oftenlittlepointindoingso. n_56 57 Thedetailsdiffer,dependinguponarchitecture,processor,anddetailsofthehardwaredesign. n_57 58 Thisdatawastakendirectlyfromthe405GPuser'smanual,referencedattheendofthischapter. n_58 59 Thisismostlyforhistoricalreasons.FromtheearlydaysofPCs,BIOSprogramsloadedonlythe firstsectorofadiskdriveandpassedcontroltoit. n_59 60 Inanacknowledgmentofthenumberofbootloadersinexistence,theYAMONuser'sguidebills itselfasYetAnotherMONitor. n_60 61 Thetermsmoduleanddevicedriverareusedhereinterchangeably. n_61 62 ThispathisusedbyRedHatandFedoradistributions,andisalsorequiredbytheFileSystem Hierarchy Standard referenced at the end of this chapter. Other distributions might use different locationsinthefilesystemforkernelmodules. n_62 63 Hosting a target board and NFS root mount are covered in detail in Chapter 12, "Embedded DevelopmentEnvironment". n_63 64 If you don't see the messages on the console, either disable your syslogd logger or lower the consoleloglevel.WedescribehowtodothisinChapter14,"KernelDebuggingTechniques". n_64 65 /proc/modulesispartoftheprocfilesystem,whichisintroducedinChapter9,"FileSystems". n_65 66 Referencetothisstandardisfoundinthe"SuggestionsforAdditionalReading,"attheendofthis chapter. n_66 67 Actually,theopen()callisaClibrarywrapperfunctionaroundtheLinuxsys_open()systemcall. n_67 68 This practice is not unique to open source. Copyright and patent infringement is an ongoing concernforalldevelopers. n_68 69 Thetermcylinderwasborrowedfromtheunitofstorageonarotationalmedia.Itconsistsofthe data under a group of heads on a given sector of a disk device. Here it is used for compatibility purposeswiththeexistingfilesystemutilities. n_69 70 Filesystemscanbemademountablebynonrootusers,aswithcdrom. n_70 71 Afileonafilesystemisrepresentedbyaninternalext2datastructurecalledaninode. n_71 72 Metadataisdataaboutthefile,asopposedtothefile'sdataitself.Examplesincludeafile'sdate, time,size,blocksused,andsoon. n_72 73 Convertingafilesysteminthismannershouldbeconsideredadevelopmentactivityonly. n_73 74 Itiscertainlypossibletomount/procanywhereyoulikeonyourfilesystem,butalltheutilities (includingmount)thatrequireprocexpecttofinditmountedon/proc. n_74 75 ThesizewasfixedinthekernelconfigurationwhenweenabledtheMTDRAMtestdeviceinthe Linuxkernelconfiguration. n_75 76 The kernel can be configured to operate with a wrong-endian MTD file system, at the cost of reducedperformance.Insomeconfigurations(suchasmultiprocessordesigns),thiscanbeauseful feature. n_76 77 Wecoverthedetailsofsymboliclinksshortly. n_77 78 WecoveredthedetailsofSystemVinitializationinChapter6. n_78 79 Webster'sdefinesnonsenseas"anideathatisabsurdorcontrarytogoodsense."Itismyopinion thatdevelopingembeddedLinuxplatformsonanon-Linux/UNIXhostisnonsensical. n_79 80 SeeSIG_KERNEL_COREDUMP_MASKin.../kernel/signal.cforadefinitionofwhichsignals generateacoredump. n_80 81 Signalsandtheirassociatednumbersaredefinedin.../asm-<arch>/signal.hinyourLinuxkernel sourcetree. n_81 82 Actually,supportfortheunderlyingenginethatcbrowserusesisintheLinuxbuildsystem. n_82 83 Seemanldconfigfordetailsoncreatingalinkercacheforyourtargetsystem. n_83 84 Sometimesanall-zerosaddressisappropriateinthiscontext.However,weareinvestigatingwhy theprogrambailedabnormally,soweshouldconsiderthissuspect. n_84 85 Seemanhostsfordetailsofthissystemadministrationfile. n_85 86 TheanalysisutilityisaPerlscriptsuppliedwiththemTRacepackage. n_86 87 A reference for the Dwarf2 Debug Information Specification is provided at the end of this chapter. n_87 88 S-record files are an ASCII representation of a binary file, used by many device programmers andsoftwarebinaryutilities. n_88 89 See the GCC manual referenced at the end of this chapter in Section 14.6.1, "Suggestions for AdditionalReading"fordetailsontheoptimizationlevels. n_89 90 Inlinefunctionsarelikemacros,butwiththeadvantageofcompile-timetypechecking. n_90 91 NotwithstandingthecommentsmadeearlieraboutKGDBoverEthernet. n_91 92 Aspointedoutearlier,thegdbremoteprotocolisdetailedinthegdbmanualcitedattheendof thischapterinSection14.6.1,"SuggestionsforAdditionalReading." n_92 93 Remember to use your cross-version of readelffor example, ppc_44x-readelf for the PowerPC 44xarchitecture. n_93 94 A reference for the Dwarf debug specification appears at the end of this chapter in Section 14.6.1,"SuggestionsforAdditionalReading." n_94 95 Ahelpfulshortcutformacrodevelopmentisthegdbsourcecommand.Thiscommandopensand readsasourcefilecontainingmacrodefinitions. n_95 96 Thesymbol_endisdefinedinthelinkerscriptfileduringthefinallink. n_96 97 As of this writing, there is a bug in gdb that prevents this technique from working properly. Hopefully,bythetimeyoureadthis,itwillbefixed. n_97 98 Anentryinthehostsystem's/etc/hostsfileenablesthesymbolicIPaddressreference. n_98 99 Allthesefilenamesareunique,sotheycanbefoundwithoutfullpathnamereferences. n_99 100 Remembertouseyourcross-versionofstrip,forexampleppc_82xx-strip. n_100 101 Itiscertainlypossibletopasstheselocationstoyourcompiler,linker,anddebuggerforevery invocation,butanygoodembeddedLinuxdistributionwillconfigurethesedefaultsintothetoolchain asaconveniencetothedeveloper. n_101 102 Of course, your compiler also needs to know the location of target files such as architecturespecificsystemandlibraryheaderfiles. n_102 103 Wewillusethetermsystemcall,butfork()inthiscontextisactuallytheClibraryfunctionwhich inturncallstheLinuxsys_fork()systemcall. n_103 104 ReferbacktoListing14-5inChapter14. n_104 105 ReferbacktoListing14-5inChapter13 n_105 106 By"homes,"wemeanapublicsourcecoderepository,suchasawebserverontheInternet. n_106 107 Byconvention,parametersinCarepassedinthesePowerPCregisters. n_107 108 TheAMCCPPC405isaperfectexampleofthis.Theinterestedreaderisencouragedtoexamine theBATregistersinthisprocessor. n_108 109 RefertotheProgrammingEnvironmentsManualreferencedattheendofthischapterfordetails ofthePowerPCDSIexception. n_109 110 Theinitialramdisk,orinitrd,wasintroducedinChapter6. n_110 111 WeintroducedtheSystem.mapfileinChapter4. n_111 112 For details of PPC assembly language syntax, see Section 16.5.1, "Suggestions for Additional Reading"attheendofthischapter. n_112 113 Eachmethodhasitsownroots.Thestructbd_infooriginatedinU-Boot,andstructbi_recordwas anattempttounifyacrossallplatforms.Botharesupportedbymanyplatforms. n_113 114 To preserve space, we temporarily removed many machine types in Figure 16-4 prior to LITE5200. n_114 115 We neglect the context switching time for interrupt processing because it is often negligible comparedtointerruptofftime. n_115 116 Robert Love explains bottom-half processing in great detail in his book Linux Kernel Development.SeeSection17.5.1,"SuggestionsforAdditionalReading,"attheendofthischapterfor thereference. n_116 117 Interestingly, there ismuchdebateonthecorrectspellingofpreemptable!Idefer tothesurvey donebyRickLehrbaumonwww.linuxdevices.com/articles/AT5136316996.html. n_117 118 AlsocalledHARDIRQs. n_118 119 SeeLinuxKernelDevelopment,referencedattheendofthischapter,tolearnmoreaboutsoftirqs. n_119 120 Seewww.rdrop.com/users/paulmck/RCU/foranin-depthdiscussionofRCU. n_120 121 WereferyouagaintoRobertLove'sbookforanexcellentdiscussionoftheO(1)scheduler,anda delightfuldiatribeonalgorithmiccomplexity,fromwhichthenotationO(1)derives. n_121 FB2documentinfo DocumentID:ooofbtools-2009-5-18-11-38-4-258 Documentversion:1 Documentcreationdate:18.05.2009 Createdusing:ExportToFB21,FBEditorv2.0software Documentauthors: honoratobonafe Documenthistory: 1.0chm->fb2 About ThisbookwasgeneratedbyLordKiRon'sFB2EPUBconverterversion1.0.28.0. Эта книга создана при помощи конвертера FB2EPUB версии 1.0.28.0 написанного Lord KiRon