[Initscripts-ng-commits] r108 - in /trunk/www/soc2006-bootsystem:
code/testhotspots.sh deliverable3/
deliverable3/debianboot.bib deliverable3/debianboot_d3.kilepr
deliverable3/debianboot_d3.tex
dvictoria-guest at users.alioth.debian.org
dvictoria-guest at users.alioth.debian.org
Sat Aug 12 10:45:05 UTC 2006
Author: dvictoria-guest
Date: Sat Aug 12 10:45:02 2006
New Revision: 108
URL: http://svn.debian.org/wsvn/initscripts-ng/?sc=1&rev=108
Log:
deliverable 3 started
Added:
trunk/www/soc2006-bootsystem/deliverable3/
trunk/www/soc2006-bootsystem/deliverable3/debianboot.bib
trunk/www/soc2006-bootsystem/deliverable3/debianboot_d3.kilepr
trunk/www/soc2006-bootsystem/deliverable3/debianboot_d3.tex
Modified:
trunk/www/soc2006-bootsystem/code/testhotspots.sh
Modified: trunk/www/soc2006-bootsystem/code/testhotspots.sh
URL: http://svn.debian.org/wsvn/initscripts-ng/trunk/www/soc2006-bootsystem/code/testhotspots.sh?rev=108&op=diff
==============================================================================
--- trunk/www/soc2006-bootsystem/code/testhotspots.sh (original)
+++ trunk/www/soc2006-bootsystem/code/testhotspots.sh Sat Aug 12 10:45:02 2006
@@ -1,4 +1,21 @@
#! /bin/bash
+# Copyright (C) 2006 Carlos Villegas and Petter Reinholdtsen
+# Full notice of GPL license can be found in the main
+# project file: checkLSB.
+#
+# Author: Petter Reinholdtsen
+# Author: Carlos Villegas
+# Created: 2006-07-26
+# Last updated: 2006-08-11
+#
+# Alter the boot process to test different hotspot combinations.
+# Among other things, the init scripts in /etc/init.d and their
+# symbolic link pool (/etc/rcX.d) are modified and packages are
+# installed according to the hotspots to be tried.
+#
+#
+# Run >>./testhotspots.sh change hotspot1 hotspot2 hotspot3 to do a dry run and
+# run >>./testhotspots.sh -d to perform the changes
date=`date +%y%m%d`
Added: trunk/www/soc2006-bootsystem/deliverable3/debianboot.bib
URL: http://svn.debian.org/wsvn/initscripts-ng/trunk/www/soc2006-bootsystem/deliverable3/debianboot.bib?rev=108&op=file
==============================================================================
--- trunk/www/soc2006-bootsystem/deliverable3/debianboot.bib (added)
+++ trunk/www/soc2006-bootsystem/deliverable3/debianboot.bib Sat Aug 12 10:45:02 2006
@@ -1,0 +1,159 @@
+ at Article{ubunto-shtodash,
+ title="Ubuntu Feature Specification: Dash as /bin/sh",
+ note="\href{https://launchpad.net/distros/ubuntu/+spec/dash-as-bin-sh}{https://launchpad.net/distros/ubuntu/+spec/dash-as-bin-sh}"}
+
+
+ at Article{bootchartd-script,
+ title="Bootchart logger script",
+ author="Ziga Mahkovec"
+ note="/sbin/bootchartd"}
+
+ at Article{bootchart-webpage,
+ title="Bootchart -- Boot Process Performance Visualization",
+ note="\newline \href{http://www.bootchart.org/}{http://www.bootchart.org/}"}
+
+ at inproceedings{bootdebian-d1,
+author="Carlos Villegas and Petter Reinholdtsen",
+title="State-of-the-Art in the Boot Process ",
+booktitle="Improve the Debian Boot Process Project of
+Google Summer of Code 2006 ",
+note="\newline \href{http://initscripts-ng.alioth.debian.org/soc2006-bootsystem/deliverable1.html}{http://initscripts-ng.alioth.debian.org/soc2006-bootsystem/deliverable1.html}",
+month=jun,
+year=2006}
+
+
+ at inproceedings{bootdebian-d2,
+author="Carlos Villegas and Petter Reinholdtsen",
+title="Benchmarking debian and first hotspots",
+booktitle="Improve the Debian Boot Process Project of
+Google Summer of Code 2006 ",
+note="\newline \href{http://initscripts-ng.alioth.debian.org/soc2006-bootsystem/deliverable2.html}{http://initscripts-ng.alioth.debian.org/soc2006-bootsystem/deliverable2.html}",
+month=jun,
+year=2006}
+
+
+
+ at Article{suseReference,
+ title="SUSE Linux Reference 10.1",
+ month=apr,
+ year=2006,
+ note="\newline \href{http://ftp.opensuse.org/pub/opensuse/distribution/SL-10.1/inst-source/docu/en/reference_en.pdf}{http://ftp.opensuse.org/pub/opensuse/distribution/SL-10.1/inst-source/docu/en/reference\_en.pdf}"}
+
+ at Article{suseREADME,
+ title="(suse 10)/etc/init.d/README -- The SUSE boot concept"}
+
+ at Article{initscripts-ng,
+ title="Debian next generation initscripts project at Alioth",
+ note="\href{http://alioth.debian.org/projects/initscripts-ng}{http://alioth.debian.org/projects/initscripts-ng}"}
+
+
+ at Article{BrasseurPreloading,
+ title="Linux Optimization",
+ author="Eric Brasseur",
+ month=may,
+ year=2005,
+ note="\newline \href{http://www.4p8.com/eric.brasseur/linux_optimization.html}{http://www.4p8.com/eric.brasseur/linux\_optimization.html}"}
+
+
+ at Article{sysvinitIRC,
+ title="\#pkg-sysvinit at irc.debian.org"}
+
+
+
+ at Article{peterSoCProposal,
+ title="Ideas for speeding up the Debian boot process",
+ month=jan,
+ year=2006,
+ note="\newline \href{http://lists.alioth.debian.org/pipermail/pkg-sysvinit-devel/2006-January/000542.html}{http://lists.alioth.debian.org/pipermail/pkg-sysvinit-devel/2006-January/000542.html}"}
+
+
+
+
+ at Article{MewburnNetBSD,
+ author="Luke Mewburn",
+ title="The Design and Implementation of the NetBSD rc.d system",
+ month=jun,
+ year=2001,
+ booktitle="Usenix Annual Technical Conference",
+ note="\newline \href{http://www.mewburn.net/luke/papers/rc.d.pdf}{http://www.mewburn.net/luke/papers/rc.d.pdf}"}
+
+ at Article{rc-scriptFreeBSD,
+ title="(freebsd/src/etc/rc script from FreeBSD",
+ month=feb,
+ year=2006,
+ note="\newline \href{http://www.freebsd.org/cgi/cvsweb.cgi/src/etc/rc}{http://www.freebsd.org/cgi/cvsweb.cgi/src/etc/rc}"}
+
+ at Article{rcorder-manFreeBSD,
+ title="RCORDER(8) in the FreeBSD System Manager's Manual",
+ month=jul,
+ year=2000,
+ note="\newline \href{http://www.freebsd.org/cgi/man.cgi?query=rcorder}{http://www.freebsd.org/cgi/man.cgi?query=rcorder}"}
+
+
+ at Article{rcS-manDebian,
+ title="rcS(5) man page in Debian Administrator's Manual",
+ month=nov,
+ year=2003}
+
+
+ at Article{ubuntu0smf,
+ title="Ubuntu and SMF",
+ note="\newline \href{http://www.calivia.com/blog/mike/solaris_smf_for_ubuntu0packages_on_opensolaris}{http://www.calivia.com/blog/mike/solaris\_smf\_for\_ubuntu\_packages\_on\_opensolaris}"}
+
+ at Article{fedora0analysis,
+ title="Fedora analysis",
+ note="\href{http://fedoraproject.org/wiki/FCNewInit}{http://fedoraproject.org/wiki/FCNewInit}"}
+
+ at Article{chkconfig-Manfedora,
+ title="CHKCONFIG(8) man page in Fedora Core 5"}
+
+ at Article{margablog-boot,
+ title="Marga's Blog -- Parallel Booting",
+ note="\newline \href{http://www.marga.com.ar/blog/index.cgi/debian/Parallel_booting.html}
+{www.marga.com.ar/blog/index.cgi/debian/Parallel\_booting.html}"}
+
+ at inproceedings{Moraes0sysvinit,
+author="Henrique de Moraes Holschuh",
+title="System Init Scripts and the Debian O.S.",
+booktitle="3rd Debian Conference",
+note="\newline \href{http://alioth.debian.org/docman/view.php/30730/38/debconf2-initscripts-bkg.pdf}{alioth.debian.org/docman/view.php/30730/38/debconf2-initscripts-bkg.pdf}",
+month=jun,
+year=2002}
+
+ at Article{readahead0aist,title="Readahead in Knoppix",
+note="\newline \href{http://unit.aist.go.jp/itri/knoppix/readahead/index-en.html}{http://unit.aist.go.jp/itri/knoppix/readahead/index-en.html}"}
+
+ at Article{readahead0die,
+ title="Readahead(2) -- Linux man page",
+ note="\newline \href{http://www.die.net/doc/linux/man/man2/readahead.2.html}{http://www.die.net/doc/linux/man/man2/readahead.2.html}"}
+
+ at Article{initng0wiki,
+ title="initng -- Next generation init system",
+ note="\newline \href{http://www.initng.org/wiki}{http://www.initng.org/wiki}"}
+
+ at Article{mandiva0cooker,
+ title="Mandrake Linux Archives: cooker at mandrivalinux.org",
+ month=oct,
+ year=2005,
+ note="\newline \href{http://archives.mandrivalinux.com/cooker/2005-10/msg00256.php}{http://archives.mandrivalinux.com/cooker/2005-10/msg00256.php}"}
+
+ at Article{initscripts0ThomasHood2feb,
+author="Jane Weideman",
+title="Ubuntu plans for init",
+note="\newline \href{https://wiki.ubuntu.com/ReplacementInit}{https://wiki.ubuntu.com/ReplacementInit}"}
+
+ at Article{ucla0sysvinit,
+ title="startpar manual",
+ note="\newline \href{www.math.ucla.edu/computing/docindex/sysvinit-man-25.html}{www.math.ucla.edu/computing/docindex/sysvinit-man-25.html}"}
+
+ at Article{man0insserv,
+ title="insserv(8) manual",
+ note="\newline \href{http://man-wiki.net/index.php/8:insserv}{http://man-wiki.net/index.php/8:insserv}"}
+
+ at Article{initscripts0Petter8feb,
+ title="Ubuntu plans for init",
+ author="Petter Reinholdtsen",
+ month=feb,
+ year=2006,
+ note="\newline \href{http://lists.alioth.debian.org/pipermail/initscripts-ng-devel/2006-February/000251.html}{http://lists.alioth.debian.org/pipermail/initscripts-ng-devel/2006-February/000251.html}"}
+
Added: trunk/www/soc2006-bootsystem/deliverable3/debianboot_d3.kilepr
URL: http://svn.debian.org/wsvn/initscripts-ng/trunk/www/soc2006-bootsystem/deliverable3/debianboot_d3.kilepr?rev=108&op=file
==============================================================================
--- trunk/www/soc2006-bootsystem/deliverable3/debianboot_d3.kilepr (added)
+++ trunk/www/soc2006-bootsystem/deliverable3/debianboot_d3.kilepr Sat Aug 12 10:45:02 2006
@@ -1,0 +1,30 @@
+[General]
+img_extIsRegExp=false
+img_extensions=.eps .pdf .dvi .ps .fig .gif .jpg .jpeg .png
+lastDocument=file:///home/carlos/Academia/debian/soc2006-bootsystem/deliverable3/debianboot_d3.tex
+masterDocument=
+name=debianboot_d3
+pkg_extIsRegExp=false
+pkg_extensions=.cls .sty .dtx
+src_extIsRegExp=false
+src_extensions=.tex .ltx .bib .mp
+
+[Tools]
+MakeIndex=
+QuickBuild=
+
+[item:debianboot_d3.kilepr]
+archive=true
+column=0
+encoding=
+highlight=
+line=0
+open=false
+
+[item:debianboot_d3.tex]
+archive=true
+column=129
+encoding=ISO 8859-1
+highlight=LaTeX
+line=125
+open=true
Added: trunk/www/soc2006-bootsystem/deliverable3/debianboot_d3.tex
URL: http://svn.debian.org/wsvn/initscripts-ng/trunk/www/soc2006-bootsystem/deliverable3/debianboot_d3.tex?rev=108&op=file
==============================================================================
--- trunk/www/soc2006-bootsystem/deliverable3/debianboot_d3.tex (added)
+++ trunk/www/soc2006-bootsystem/deliverable3/debianboot_d3.tex Sat Aug 12 10:45:02 2006
@@ -1,0 +1,236 @@
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+% Improve the Debian Boot Process.
+% Google Summer of Code 2006
+% Second Delivery (D2).
+% o DEBIAN
+% o Carlos Villegas
+% o Petter Reinholdtsen
+% o Created: 30-June-2006
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+
+\documentclass[a4paper,10pt]{article}
+
+\bibliographystyle{plain}
+\usepackage{subfigure}
+\usepackage[dvips]{graphicx}
+\usepackage{hyperref}
+
+
+ \usepackage[latin1]{inputenc}
+ \usepackage{fullpage}
+ \usepackage{color}
+ \usepackage{array}
+ \usepackage{longtable}
+ \usepackage{calc}
+ \usepackage{multirow}
+ \usepackage{hhline}
+ \usepackage{ifthen}
+
+
+\parskip 3mm
+\parindent 0mm
+
+%%\usepackage{pdfdraftcopy}
+%opening
+\title{Improve the Debian Boot Process\\
+Google Summer of Code 2006 \\
+\vspace{4mm}
+Implementation of hotspots\\
+Third Deliverable\\
+\vspace{4mm}}
+
+\author{\emph{student}: Carlos Villegas (Carlos.Villegas at nuim.ie)\\
+\emph{mentor}: Petter Reinholdtsen (pere at hungry.com)}
+
+
+
+\begin{document}
+
+\maketitle
+
+
+\section{Introduction}
+
+In this deliverable we intend to show the results of testing the remaining hotspots and to explore briefly the interactions between them.
+
+
+Several approaches have been tried along the past two months of the Summer of Code together with some patches, scripts and bug reports. Now it is time to review the effect of the remaining hotspots and try them together to see the total time improvement in the boot time.
+
+First, in section~\ref{sec:hotspots}, we will ennumerate the different hotspots, present their testing procedure and their individual effect on boot time. Afterwards, in section ~\ref{sec:combinedhotspots}, we will try tem together in some combinatios and show the total time improvement.
+
+Discussion on the project may be followed in the \href{http://lists.alioth.debian.org/mailman/listinfo/initscripts-ng-devel}{initscripts-ng-devel mailing list} and the channel \#pkg-sysvinit in \emph{irc.debian.org}.
+
+\section{Individual Hotspots}
+\label{sec:hotspots}
+
+\begin{table}[t]
+\begin{center}
+\begin{tabular}{cccc}
+ \hline
+ % after \\: \hline or \cline{col1-col2} \cline{col3-col4} ...
+Order & \textit{Hotspot} & Marga's time\cite{margablog-boot} & our time \\
+ \hline
+1 & dash & 6 & 3\\
+2 & hwclock & 6 & 2 \\
+3 & depmod & 2 & 2\\
+4 &network & 2 & 0 \\
+5 & reorder & 2 & -- \\
+6 & parallel & -- & 2 \\
+7 & discover & -- & 0 \\
+8 & preloading & --& 0 \\
+ \hline
+ \end{tabular}
+ \end{center}
+ \caption{\textit{hotspot} time reduction}
+ \label{tab:hotspots}
+\end{table}
+
+As any changes resulting from this project may be implemented in Etch+, we've used the Debian unstable release -- Sid --, to test the hotspots. According to the discussion in ~\cite{bootdebian-d2}, we may consider two different metrics:
+
+\begin{description}
+\item[startup time] - from the point of view of the user from turning on the computer until KDE becomes usable, and
+\item[sysvinit time] - from the moment /sbin/init takes control of boot process until KDE becomes usable.
+ \end{description}
+
+We will use the second metric because it is the one that can be influenced by modifying the sysvinit boot process and it is a subset of the first metric.
+
+\subsection{dash}
+
+Use dash instead of bash reduces the boot time from around 2 to 6\cite{margablog-boot} seconds depending on the method and system used. It seems to be the \textit{hotspot} with the bigger influence. It may be implemented in two ways:
+
+\begin{enumerate}
+\item by linking /bin/sh to /bin/dash, and
+\item by changin the init scripts to use dash.
+\end{enumerate}
+
+The first approach caused a 4 second improvement while the second approach a 2 second improvement in our test system. As a remark, linking /bin/sh to /bin/dash has been already implemented by Ubuntu~\cite{ubunto-shtodash}
+
+To first approach is implemented as follows:
+
+\textit{apt-get install dash
+dpkg-divert --add /bin/sh
+mv /bin/sh /bin/sh_old
+cd /bin
+ln -s dash sh}
+
+and removed as:
+
+\textit{dpkg-divert --remove /bin/sh
+rm /bin/sh
+mv /bin/sh_old /bin/sh
+apt-get remove --purge dash
+apt-get install --reinstall bash}
+
+For the second approach, the test consisted of changing the shell used for all the scripts in /etc/init from /bin/sh to /bin/dash
+
+
+
+ It depends on
+around 6 seconds in the tests made by Margarita Manterola\cite{margablog-boot}. This seems to be due to the smaller size of dash. The time reduction for our tests was only of 3 seconds.
+
+
+\subsection{hwclock}
+
+Setting up the hardware clock in the background was proposed by Margarita Manterola and she obtained a reduction of 6 seconds in the boot time\cite{margablog-boot}. Nevertheless, running hwclok on the background may cause problems with processes that can get confused if the time changes while they are running. This is the case of startpar. To overcome this problem, startpar doesn't run the scripts with sh extension in parallel.
+
+We tried to load the hwclock on the background with a 2 seconds reduction in the boot time.
+
+
+
+
+\subsection{Remove depmod from the boot process}
+
+Depmod is not used any more in sid's module-init-tools and this is one of the hotspots. This was removed as a result from a discussion started by Margarita Manterola in debian-devel. Just by adding it again from an old script, the 2 second reduction from hwclock was achieved. 2 seconds were gained by Margarita Manterola\cite{margablog-boot} as well.
+
+
+
+\subsection{Set up the network in the background}
+
+Setting up the network in the background was proposed as a promising hotspot based on the results from Margarita Manterola~\cite{margablog-boot} with a 2 second reduction. This hotspot was tried by modifying the networking init script but no time difference was noticed.
+
+One problem with this hotspot is related to the network status return after running "ifup -a". In the script "ifup -a" is inside an if statement such that the status return of the network can be handled properly. This makes no sense if we send it to the background. Therefore, if this approach is widely accepted, the current init script will have to be modified.
+
+
+\textbf{LSB-compliance and reordering bootscripts}
+
+LSB compliance in the init scripts can be used to reorder the scripts as shown in the SUSE boot with insserv~\cite{suseReference}. Lintian rules should be considered as well during the implementation. Just by reordering some processes around 2 seconds could be gained\cite{margablog-boot}. This hotspot has not been tested but a script to check LSB compliance was created and the ordering script of insserve is going to be modified.
+
+
+\subsection{Parallel execution of boot scripts}
+We tried parallel execution using startpar. In order to avoid unmet dependencies during the boot time, insserv was used previously to order the scripts. The small time reduction seems to be due to a deficient ordering of the init scripts by insserv as it may be observed by comparing the init scripts order before and after ordering with insserv.
+
+In the new order of the initscripts it can be observed that:
+\begin{itemize}
+\item there are repeated versions of one initscript in the symbolic link farms (/etc/rcS.d and /etc/rc2.d) like udev,
+\item some scripts with dependency information (like udev and alsa-utils) were not ordered properly.
+\end{itemize}
+
+To correct this behavior requires some modifications to the code for ordering the initscripts in insserv. This would be the next step in the project (to implement a promising hotspot) just after dependency information (LSB-compliance) can be checked.
+
+
+
+\subsection{Remove discover}
+The discover script was removed as udev is already doing the same job. This change is valid for 2.6 kernels as older kernels don't support udev. We noticed no change on the boot time by removing discover. This should be hardware dependent.
+
+
+\subsection{Preloading}
+
+Having parts of the program already in memory can improve execution speed. Nevertheless, with parallel execution, preloading should be adjusted not to hinder other programs memory requirements~\cite{BrasseurPreloading} (possibly solved with dynamic preloading).
+
+There are different programs for preloading. There is a debian package of a dynamic readahead called preload. It was tested but the default installation showed no improvement in the boot time.
+
+
+\subsection{Improve CPU usage with X desktop manager}
+
+Improve the use of resources when starting the desktop manager (like KDE or Gnome) like with ELF prelinking,
+
+\subsection{Make the boot less verbose}.
+
+Making the boot print less messages to the screen should decrease the boot time. This approach probably requires to modify most scripts although the issue of deciding the information that should be showed would remain open.
+
+\subsection{Use internal functions in the init scripts}
+Rewritting slow shell scripts to use internal function instead of external programs is another option.
+
+
+\subsection{Use ELF prelinking} The use of ELF prelinking could help with the programs that need to link to many libraries, e.g. KDE.
+
+
+
+\section{Hotspot combination}
+
+\begin{table}[t]
+\begin{center}
+\begin{tabular}{cccc}
+ \hline
+ % after \\: \hline or \cline{col1-col2} \cline{col3-col4} ...
+\textit{Hotspot} & Comb.1 & Comb.2 & Comb.3 & Comb.4 & Comb.5 & Comb. 6\\
+ \hline
+dash & X & X & X & X & X & \\
+hwclock & X & & X & X & & X \\
+network & X & & X & X & & \\
+reorder & X & X & X & X & X & \\
+parallel & & X & X & & X & X \\
+preloading & & X & X & X & X & X \\
+ \hline
+time red. & 4 & 6 & 6 & 6 & 6 & 4 \\
+ \hline
+ \end{tabular}
+ \end{center}
+ \caption{Time reduction with \textit{hotspot} combination}
+ \label{tab:hotspots}
+\end{table}
+
+
+The different \textit{hotspots} were tested in combination with others to try to find both, the interactions among them and the best probable combination.
+
+Automatic script...
+
+
+\section{Conclusions and Future Work}
+
+
+
+Discussion on the project may be followed in the \href{http://lists.alioth.debian.org/mailman/listinfo/initscripts-ng-devel}{initscripts-ng-devel mailing list} and the channel \#pkg-sysvinit in \emph{irc.debian.org}.
+
More information about the Initscripts-ng-commits
mailing list