source: soft/build_system/build_system/mkcd/tags/V3_8_0_1mdk/mkcd.old.xml @ 1

<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook V4.1//EN"
"/usr/share/sgml/docbook/sgml-dtd-4.1/docbook.dtd">
<article>
  <title>mkcd-howto</title>

  <sect1>
    <title>What does mkcd ?</title>
    
    <para>
      mkcd is a script written in Perl and designed to make the design of CD images of a linux-mandrake distribution easier.
    </para>

    <para>
      mkcd uses as input a set of RPMs and SRPMs, and other files like the image of a boot disk and the
      install program; it outputs iso 9660 images. Its action is parametrized by a configuration file which
      basically tells what belongs to each CD. Some checks are performed, and finally the ISO images are
      created.
    </para>
    
    <para>
      The checks mainly involve dependencies between RPM packages, using rpmtools (see rpmtools manpage). 
    </para>
    
    <para>
      mkcd uses a working directory to perform its task, with some standard subdirectories:
    </para>

    <itemizedlist>      
      <listitem>
        <para>input : the configuration files, together with the files listing the packages on each CD. for example, for a
          8.0 distrib, one 8.0.conf file, and files 8.0-1, 8.0-2, 8.0-3, 8.0-com1, 8.0-com2, 8.0-com3,
          8.0-ViaVoice, which are files describing the packages of the corresponding CDs.
        </para>
      </listitem>
      <listitem>
        <para>build : where the file hierachy of each CD is written</para>      
      </listitem>      
      <listitem>
        <para>iso : where the isos are to be (finally) written</para>   
      </listitem>      
      <listitem>
        <para>config : some files produced by mkcd describing what it has actually found and will go in the CDs</para>  
      </listitem>      
      <listitem>
        <para>tmp : temporary files, including the list of the dependencies between packages (depslist.ordered) and the
          file containing all the headers of the rpms, manipulated by rpmtools and by the installation process</para>   
      </listitem>
    </itemizedlist>

    <para>
      Usually, a "log" subdirectory can also be found, and when there is the need to make a
      local copy of the files which go in the distrib (rpms, srpms...), they are stored in a subdirectory named
      "pieces".
    </para>

  </sect1>
  <sect1>
    <title>Checks performed by mkcd</title>
    
    <para>
      In order to create the filesystem of each CD, mkcd first builds intermediate lists of packages, from the
      lists the user provided. These lists may differ from the user-provided ones because of the checks
      performed by mkcd against
    </para>

    <itemizedlist>      
      <listitem>
        <para>
          dependency problems caused by missing packages
        </para>
      </listitem>
      <listitem>
        <para>
          corrupted RPMs
        </para>
      </listitem>
      <listitem>
        <para>
          mismatching SRPM and RPM version numbers
        </para>
      </listitem>
      <listitem>
        <para>
          multiple copies of each package (different versions for example)
        </para>
      </listitem>
    </itemizedlist>

  </sect1>
  <sect1 id=cmdline>
    <title id=cmdline.title>mkcd command-line options, examples of use</title>

    <para>
      <table frame="all"><title>Options relative to lists</title>
        <tgroup cols="2" align="left" colsep="1" rowsep="1">
          <tbody>
            <row>
              <entry>-l, --lists-check</entry>
              <entry>check the lists</entry>
            </row>
            <row>
              <entry>-f, --fast</entry>
              <entry>fast mode.</entry>
            </row>
            <row>
              <entry>-d, --depslist-creation</entry>
              <entry>rebuild the depslist.ordered file before checking the list.</entry>
            </row>
          </tbody>
        </tgroup>
      </table>
    </para>

    <para>
      <table frame="all"><title>Other common options</title>
        <tgroup cols="2" align="left" colsep="1" rowsep="1">
          <tbody>
            <row>
              <entry>--distrib</entry>
              <entry>specify the target distrib.</entry>
            </row>
            <row>
              <entry>-a, --auto</entry>
              <entry>Automated mode, add dependencies.</entry>
            </row>
            <row>
              <entry>-m, --make  &lt;cds number> </entry>
              <entry> Build the CDs.</entry>
            </row>
            <row>
              <entry> -c, --catto  &lt;log file> </entry>
              <entry> Log file.</entry>
            </row>
            <row>
              <entry> -o, --outname  &lt;tag> </entry>
              <entry>Tag added to the VERSION file.</entry>
            </row>
          </tbody>
        </tgroup>
      </table>
    </para>
    
    <para>
      To make a distrib, one usually proceeds in two steps: building the package lists (and checking dependencies)
      and building the CDs.
    </para>

    <sect2>
      <title>Building the lists of packages</title>
      <para>
        The first one consists in building the list of packages for each CD (in the config/ subdirectory) and the
        dependencies list; this list enables to ensure that there are no required packages missing.
      </para>

      <para>
        Assume we are in the working directory of mkcd (with its subdirectories "tmp", "input"...)
      </para>
      
      <para>
        <programlisting format="linespecific">
        mkcd --distrib "80" -c log/freq.log -l input/8.0-freq.conf 
        </programlisting>
      </para>
      
      <para>
        builds in "config/" the lists of packages for each CD, following the rules of the configuration file
        "input/8.0-freq.conf", using the logfile "log/freq.conf". For each CD, several files are created. Their names
        begin with the name of the CD (for example, 8.0-freq-1 (CD 1 of the 8.0-freq distrib) or 8.0-ViaVoice (CD
        ViaVoice of the 8.0 distro)). For a CD named NAME, you'll find in config after the -l stage:
      </para>


      <table frame="all"><title>Files created in config/</title>
        <tgroup cols="2" align="left" colsep="1" rowsep="1">
          <tbody>
            <row>
              <entry>NAME</entry>
              <entry>raw names of the rpms on the CD, without version numbers and .rpm extension</entry>
            </row>
            <row>
              <entry>NAME.rpm</entry>
              <entry>complete names of the rpms of the CD</entry>
            </row>
            <row>
              <entry>NAMEs</entry>
              <entry>raw names of the srpms associated with the rpms of the CD</entry>
            </row>
            <row>
              <entry>NAME.srpm</entry>
              <entry>complete names of the srpms</entry>
            </row>
          </tbody>
        </tgroup>
      </table>
      
      <para>
        Some other lists may be generated if the config file in input/ requires to split the srpm CDs. See the
        <quote><link linkend=config endterm="config.title"></link></quote> section.
      </para>
      
      <para>
        <programlisting format="linespecific">
          mkcd --distrib "80" -c log/freq.log -d input/8.0-freq.conf 
        </programlisting>
      </para>

      <para>
        builds in "tmp/" the dependencies list depslist.ordered, and a file containing all the headers of the rpms of
        the distrib, hdlist (in a gzipped form). Checks are performed against missing packages. "-c" option works as
        before.
      </para>

      <para>
        -d and -l options can be combined; thus one often writes
      </para>

      <para>
        <programlisting format="linespecific">
          mkcd --distrib "80" -c log/freq.log -dl input/8.0-freq.conf 
        </programlisting>
      </para>

      <para>
        to perform the two operations in one single pass.
      </para>

      <para>
        The name associated with the option --distrib is used to <!-- ???????????????????????? -->
      </para>

      <para>
        After this step, you can take a look at the logfile to see a list of the problems mkcd has found. Here is an
        example of the sections of this file for a distribution with 2 CDs (namely the freq whose lists were build
        with the previous command)
      </para>
    
      <para>
        <programlisting format="linespecific">
          DUPLICATE RPM (dir where the rpms are taken. see 
             configuration file)
          (list of RPMs duplicate, i.e. for which there are at least
             two different versions available in dir)
          
          DUPLICATE SRPM (dir)
          (list of duplicate SRPMS)

          FAILING DEPS
          (list of required rpms which are missing)
          
          NOT IN REPOSITORIES CD 1
          <!-- ?????????????? -->

          SRPMS NOT IN REPOSITORY CD 1
          <!-- ???????????? -->

          BAD SRPMS VERSION CD 1
          (list of corrupted rpms on CD 1. See configuration file for the
             splitting of rpms between the different CDs.)
          
          DUPLICATE RPMS CD 1 
          (list of duplicate rpms on CD 1)

          NOT IN REPOSITORIES CD 2
          
          SRPMS NOT IN REPOSITORY CD 2
          
          BAD SRPMS VERSION CD 2
          
          DUPLICATE RPMS CD 2

          PACKAGES NOT IN LIST
          <!-- ?????????????????????????? -->
        </programlisting>
      </para>

      <para>
        Example of falling dependencies section:
      </para>

    <para>
      <programlisting format="linespecific">
          FAILING DEPS
          rute ghostscript-module-X
          gv ghostscript-module-X
        </programlisting>
    </para>

    <!-- ??????? signification : le premier package est celui qui a besoin du second qui est manquant, je suppose. -->

    <para>
      You can see that the names of the packages are displayed without the version numbers.
    </para>

    </sect2>
    <sect2>
      <title>Building the CD images</title>

      <para>
        This is performed as following:
      </para>
      
      <para>
        <programlisting format="linespecific">
          mkcd --distrib "80" -c /log/freq_build.log -o Traktopel-Freq1-i586 -m 1,2 input/8.0-freq.conf 
        </programlisting>
      </para>
      
      <para>
        The options regarding the configuration file and the log file work as before; here we build CDs 1 and
        2 (option "-m 1,2"). The name and the place where the isos are created depends on the argument of the
        "-o" option and on the name of the distrib in the config file (see <quote><link linkend=config
        endterm="config.title"></link></quote> section), in that
        case "8.0-freq". The first iso will be called "1-8.0-freq-Traktopel-Freq1-i586.iso" and the second
        "2-8.0-freq-Traktopel-Freq1-i586.iso" (these names are a bit redundant in the present case). They are
        put in the "iso/8.0-freq" subdirectory of the working dir; the "iso" subdirectory must be created
        before the command is issued.
      </para>
          
    </sect2>
  </sect1>
  <sect1>
    <title>Config files, packages lists (input/ subdirectory)</title>
    
    <para>
      The files found in that subdir fall into two categories: config files and packages lists. The config
      file gives to mkcd the global structure of the distribution, but does not include the list of the
      packages on each CD. Those lists are stored in separate files corresponding to CDs.
      For example, for a freq distrib, one finds the following files:
    </para>
    
    <itemizedlist>      
      <listitem>    
        <para>
          8.0-freq.conf: the main config file
        </para>
      </listitem>
      <listitem>    
        <para>
          8.0-freq-1: list of packages for CD #1
        </para>
      </listitem>
      <listitem>    
        <para>
          8.0-freq-2: list of packages for CD #2
        </para>
      </listitem>
    </itemizedlist>      
    
    <para>
      We'll see that the names of the package list files must match those given in the main config file.
    </para>
    

    <sect2 id=config>
      <title id=config.title>Configuration file</title>
      <para>
        A config file consists in a general section providing some global information regarding the distrib,
        and in one more specific section for each CD.
      </para>
      
      <sect3>
        <title>General section</title>
        <para>
          The general section looks like this: 
        </para>
        
        <para>
          <programlisting>
            8.0
            -r pieces/frozen/extra/RPMS -s pieces/frozen/extra/SRPMS \
            -r pieces/frozen/RPMS -s pieces/frozen/SRPMS \
            -r /mnt/BIG/dis/cdcom/traktopel/cd1/Mandrake/RPMS \
            -r /mnt/BIG/dis/cdcom/traktopel/cd2/Mandrake/RPMS \
            -r /mnt/BIG/dis/cdcom/traktopel/ViaVoice/ViaVoice/common/Mandrake/RPMS \
            -r /mnt/BIG/dis/cdcom/traktopel/ViaVoice/ViaVoice/us/Mandrake/RPMS \
            -r /mnt/BIG/dis/cdcom/traktopel/ViaVoice/ViaVoice/gr/Mandrake/RPMS 
            -r /mnt/BIG/dis/cdcom/traktopel/ViaVoice/ViaVoice/fr/Mandrake/RPMS 
            -k 8.0-kill 8.0-1,680000000 8.0-2,270000000 \
            8.0-3 8.0-com1 8.0-com2 8.0-ViaVoice 8.0-com3
          </programlisting>
        </para>
        
        <para>
          Apart from the "8.0" line, everything must be on the same line (linebreaks were added here to
          improve readability). Following -r options are the paths of the RPMs of the distribution; the paths
          following -s options are for srpms. Be aware that in case there is more than one file for a given
          package, the file which lies in the first directory in the list will be chosen, regardless of its
          version number: thus for example updates directories must be encountered <emphasis> before
          </emphasis> regular directories in the the list of rpm dirs.
        </para>
        
        <para>
          Normally srpms must be in one-to-one correspondence with rpms, with matching version numbers; but some srpms
          are really not useful (in the case of documentation packages for example, rpms and srpms are nearly the
          same). Thus you may want to exlude some srpms from this correspondence checks. This is performed by the -k
          option. It is followed by the name of a file in the input/ directory wich contains the names of the packages
          concerned, with the same conventions as in the package list files.
        </para>
        
        <para>
          This section ends with the names of the package list files, optionally followed by a list of numbers. These
          numbers enable to split the list of srpms corresponding to a CD (or a rpm list) in smaller subsets. For
          example, 8.0-3,200000000,300000000 tells mkcd to split the set of srpms corresponding to the list of rpms
          stored in the file "8.0-3" into two (possibily three) lists, the first one of total size &les; 200 MB (file
          8.0-3.srpm.1 in config/ dir), the second one of total size &les; 300 MB (file 8.0-3.srpm.2), and the third one
          (8.0-3.srpm.3) with the remaining rpms if there are any. In the above example, "8.0-1,680000000" enables to ensure
          that the source CD corresponding to CD 1 is not oversized.
        </para>
        
        <para>
          The name at the beginning of the section is the product name ("8.0" in the above example). It is used in the
          names of the isos (see <quote><link linkend=cmdline endterm="cmdline.title"></link></quote>), and
          in the names of the subdirectories in build/, tmp/, 
          iso/ (in these three directories the files created are stored in a subdirectory specific to the product you're
          building).
        </para>
        
      </sect3>
      <sect3>
        <title>CD section</title>
        <para>
          A section corresponding to a CD looks like this :
        </para>
        
        <para>
          <itemizedlist>
            <listitem>
              <para>
                A first "header" line with some fixed fields:
                
                <programlisting>
                  1 CD1 1DL8000 Installation CD (x86)
                </programlisting>
              </para>
            </listitem>
            <listitem>
              <para>
                then a sequence of (mkcd built-in) commands, one per line. For example:
                
                <programlisting>
                  generic -d Mandrake/RPMS -r pieces/frozen/extra/RPMS \
                  -r pieces/frozen/RPMS 8.0-1.rpm
                  advertising -d pieces/pics/advert/advertising \
                  1-thanks.png 2-internet.png 3-graphic.png 4-develop.png \
                  5-contcenter.png 6-user.png 7-server.png 8-games.png \
                  9-MDKcampus.png 10-MDKexpert.png
                  advertising -l fr -d pieces/pics/advert/advertising.fr \
                  1-thanksFR.png 2-internetFR.png 3-graphicFR.png \
                  4-develop.png 5-contcenterFR.png 6-userFR.png 7-serverFR.png \
                  8-gamesFR.png 9-MDKcampusFR.png 10-MDKexpertFR.png
                  installation_8 -b cdrom.img -i pieces/install 1/Mandrake/RPMS \
                  2/Mandrake/RPMS2
                </programlisting>
              </para>
            </listitem>
          </itemizedlist>
        </para>

        <para>
          there is no newline between those two parts.
        </para>
        
        <sect4>
          <title>First line of a CD section</title>

          <para>
            <table frame="all"><title>Fields in the first line</title>
              <tgroup cols="2" align="left" colsep="1" rowsep="1">
                <tbody>
                  <row>
                    <entry>1</entry>
                    <entry>cd number : number identifying the CD (internal to mkcd)</entry>
                  </row>
                  <row>
                    <entry>CD1</entry>
                    <entry># of the CD in the distrib</entry>
                  </row>
                  <row>
                    <entry>1DL8000</entry>
                    <entry>serial # of the CD</entry>
                  </row>
                  <row>
                    <entry>Installation CD (x86)</entry>
                    <entry>name of the CD</entry>
                  </row>
                </tbody>
              </tgroup>
            </table>

          </para>
          
          <para>
            The difference between distrib numbers and internal numbers arises from the fact that one configuration file
            can be used to build different releases of the same distribution (e.g. for different languages, or different
            platforms). In that case there are more CDs in the configuration file than in each release, and one must make
            a difference between numbers for mkcd to identify CDs and public numbers.
          </para>
        </sect4>

        <sect4>
          <title>mkcd commands avaliable</title>

          <table frame="all"><title>Commands available</title>
            <tgroup cols="2" align="left" colsep="1" rowsep="1">
              <tbody>
                <row>
                  <entry>generic -r path/RPMS -r path2/RPMS ... -r path(n)/RPMS -d dest listname.rpm</entry>
                  <entry>looks for the rpms listed in listname.rpm in the directories listed with the -r option, and copies them in 
                    build/product_name/cd_number/dest/</entry>
                </row>
                <row>
                  <entry>cp src dest</entry>
                  <entry>copies files from src/ to build/product_name/cd_number/dest</entry>
                </row>
                <row>
                  <entry>cdcom -d dest/ path/</entry>
                  <entry>looks for a RPMS directory in path, and copies its content in  build/product_name/cd_number/dest/ 
                    <!-- ??????????????? EST-CE QUE DEST DOIT EXISTER SUR LE CD ? -->
                  </entry>
                </row>
                <row>
                  <entry>advertising -l LANGUAGE -d path/ img1.png ... img(n).png</entry>
                  <entry>copies the images img1.png ... img(n).png  found in path/ in
                    build/product_name/cd_number/Mandrake/share/advertising_LANGUAGE (for example LANGUAGE=fr, ru ...). These are
                    the advertising messages that will be displayed during the installation process in the language chosen by the
                    user.</entry>
                </row>
                <row>
                  <entry>installation_8 -b image -i install_path/ path1 ... path(n)</entry>
                  <entry>copies the content of install_path/ in build/product_name/cd_number/, computes dependency files for rpms
                    found in directories path1 ... path(n), and uses the boot floppy disk image image to make a bootable CD.
                    This command is of course used for install CDs, but you can call it for whatever other purpose.
                    <!-- ???????????? QUE DEVIENNENT LES DEPENDANCES QUI SONT CALCULEES ? EST-CE LE FICHIER DEPSLIST QUE L'ON -->
                    <!-- RETROUVE DANS build/product_name/cd_number/Mandrake/base ? --></entry>
                </row>
              </tbody>
            </tgroup>
          </table>
        </sect4>
      </sect3>
      </sect2>
    <sect2>
      <title>Package list files</title>
      <para>
        In package list files, the names of the packages appear without version numbers, as following:
      </para>
      
      <para>
        <programlisting>
          Aurora
          Aurora-Monitor-NewStyle-Categorizing-WsLib
          Aurora-Monitor-NewStyle-WsLib
          Aurora-Monitor-Traditional-Gtk+
          Aurora-Monitor-Traditional-WsLib
          Bastille
          ...
        </programlisting>
      </para>
    </sect2>
  </sect1>
</article>