Disclaimer:
These pages about different languages / apis / best practices were mostly jotted down quckily and rarely corrected afterwards.
The languages / apis / best practices may have changed over time (e.g. the facebook api being a prime example), so what was documented as a good way to do something at the time might be outdated when you read it (some pages here are over 15 years old).
Just as a reminder.

Installation of scripts and binaries

organizing the release of your programs and scripts onto a server

Scribblings on CVS...

Installation of scripts and binaries

Any feedback is highly appreciated, you can contact me at thee email adress at the footer of this page.

1 BACKGROUND

Being at a company where we were not allowed to install new software, I had to figure out a way to get large amount of shell scripts and software under control. The shell scripts was installed on several different servers, and if someone needed to edit any of the scripts, the person would logon directly onto the server and start edit the script with VI !!! If the person pressed the wrong buttons, well, nothing else to do then create the script from scratch or figure out who is responsible for retrieving backups.
The source files for the binarys where not in a much better state, they were spread out over user accounts, and the situation was not better when the user had several different versions of the source files and could not remember which one he used to create the binary that was used in production.
Makefiles was a thing unheard of, every time the source should be compiled it was a nice guess of "uh....which compiler did we use...eh...with which flags...erm....hang on..."

To get to the point; it was a big mess. The first task I got when I was hired was to document the software we had (hurray, that is such a fun task), to get to know the software before I started editing it.
Oh, well, since I did not want to do any modifications to any script or such like, I mean everything was just a big card house waiting to fall down, I did the following:

  • Threw all code into CVS, I diveded the scripts/sources into subdirs depending on what purpose they were used for. This sounds easy but what actually the hardest bit; trying to find all scripts, and trying to figure out which ones that actually was used, where scripts where installed (in some really strange places they have hidden productive scripts), it was not that easy. Even now, one year after I stumble on some old forgotton productive script.
  • Created a script that will transfer the file from the workstation to the correct server. Since the scripts and binaries were spread out all over the place, it would take ages if the user has to find out which server and which directory the script should be transfered to.

    Below is a document that describes the procedure of the installation. All directory and file names are made up and should obviously be someting more intelligent than "project1", "theFiles", etc.
    Here, is a description of how the installation script works, the source to the installation script as well as an example how the Makefile for the binaries look like.

    here we go ...

    To install modified scripts we use an installation procedure, rather than doing rcp/ftp to a server.
    That means that we develop and do testing on our own machines, and use the installation procedure (described below) to install the scripts/binaries.

    The advantages are that:

    • it is easier, faster and more reliable than doing ftp/rcp
    • it makes sure the script is installed on the right machine in the right place with the right username and the correct rights
    • everyone will be in the loop what changes that are taking place to the scripts/programs (via mail)
    • we will get a log when what was installed. Hence, if we find out that there were some problems say 2 weeks ago, we can have a look in the log to find out if we did any changes at that time that could have caused the problem

    2 SETUP

    The following things need to be done first:
    You should use CVS. You should have the CVS files checked out in the directory ~/theFiles
    You need to have ~/theFiles/tools/installation/install.ksh in your path, what I found useful was to add a symlink from ~/bin/install.ksh to the above script.

    3 INSTALLATION

    We have two different procedures, one for installing single scripts, and one for installing binaries. The reasons for this separation are described below.

    3.1 INSTALLATION OF SCRIPTS

    When you have changed your scripts, tested them and put them back into the repository, it is time to install them.
    To do this, you will call install.ksh (as yourself, on your machine). To describe how it is done, we do an example where we first modify and then install the your script.

    3.1.1 Modifying a script

    You are user "user12" and you want to change a script that is used for generating some statistics. The script you want to change is called "gen_stats.sh" and on the production machine "hobbes" it is located in /opt/statscripts/

    First you make sure that you have the latest file in your home directory: 

     % cd ~/theFiles/project1/stat_scripts
 % cvs st gen_stats.sh
 ===================================================================
 File: gen_stats.sh   Status: Up-to-date
 
    Working revision:    1.2
    Repository revision: 1.2     /cvs/data/theFiles/project1/stat_scripts/gen_stats.sh,v
    Sticky Tag:          (none)
    Sticky Date:         (none)
    Sticky Options:      (none)
 %
    Obviously you have got the latest version of the file, and you have not changed the file.
    You can start edit the file. After you have modified the file, you can check the status once again: 
     % cvs st gen_stats.sh
 ===================================================================
 File: gen_stats.sh   Status: Locally Modified
 
    Working revision:    1.2
    Repository revision: 1.2     /cvs/data/theFiles/project1/stat_scripts/gen_stats.sh,v
    Sticky Tag:          (none)
    Sticky Date:         (none)
    Sticky Options:      (none)
 %
    As you see, your file is modified. When you have tested the script you are happy with your modifications, you can put them back into the repository. 
     % cvs commit -m "Made all values be in percentage" gen_stats.sh
 %
    You have now checked the file back into the repository, and added the comment "Made all values be in percentage" to the cvs log. If you do a 
     % cvs log gen_stats.sh
 RCS file: /cvs/data/theFiles/project1/stat_scripts/gen_stats.sh,v
 Working file: gen_stats.sh
 head: 1.3
 branch:
 locks: strict
 access list:
 symbolic names:
 keyword substitution: kv
 total revisions: 3;     selected revisions: 3
 description:
 ----------------------------
 revision 1.3
 date: 2002/11/18 18:22:03;  author: user12;  state: Exp;  lines: +18 -1
 Made all values be in percentage
 ----------------------------
 revision 1.2
 date: 2002/11/11 08:12:43;  author: user12;  state: Exp;  lines: +30 -11
 Added ability to run the script in a testenv
 ----------------------------
 revision 1.1
 date: 2002/11/11 08:09:44;  author: user12;  state: Exp;
 *** empty log message ***
    you will see the log for this file, and which changes that have been made to it.

    3.1.2 Install the script

    For single scripts we do not need tagging, we use the $Id:$ keyword (if you create a new script you should put it somewhere at the top of the script), that will tell which version of the script that is in use.

    Back to our example, we will now install the script. Notice the use of the variable $PWD. 

       % whoami
   user12
   % hostname
   calvin
   % cd ~/theFiles/project1/stat_scripts/
   % install.ksh -m "Made all values be in percentage" $PWD/gen_stats.sh
   Enter password for user prod87: ********
   FTPing
     calvin:/home/user12/theFiles/project1/stat_scripts/gen_stats.sh
   to
     hobbes:/opt/statscripts/
   Installation completed.
   %
    The file has been transfered to the machine hobbes (using the username prod87 for the logon to that machine).
    Now the file should be installed properly, please login to the production machine and verify that the execute bit is set, and that the file has been properly transfered. 
       % hostname
   hobbes
   % cd /opt/statscripts
   % ls -l gen_stats.sh
   -rwxr-xr-x   1 prod87     cax           6407 Nov 18 18:55 gen_stats.sh
   %
    A mail is sent out to those concerned and there is also a line is added to the log located in ~prod87/log/installed.txt. That logfile is quite redundant (if you keep your emails that is) and will probably not be used in future. The idea was to auto-commit that log to CVS, but it seems sort of redundant.

    If you do not receive an email when a script is installed, you can add your email address to install.ksh.

    3.2 INSTALLATION OF BINARIES

    The installation of binaries is a bit different for practical reasons.
    What differs a binary from a script is that:
    • Binaries consists of several source files, we need to be able to track exactly which version of each source file was used to build a specific binary
    • We need to be able to see which version of a binary that is installed
    To map a specific set of source files to a specific binary, we use the ability to tag files in CVS.
    Tags will not be described here, refer to documentation for CVS, from here on it is assumed that the reader is familiar with the concepts of tags.

    When you have modified your set of source files, tested the binary and commited the source files to cvs, it is time to release the binary.
    To do this you shall first tag the set of source files. You tag them with the name "rel_YYMMDD", e.g. "rel_021229".
    For example 

       % cd ~/theFiles/project1/calculation/
   % cvs tag rel_021229 .
   %
    We normally have only one tag per day.
    If you do need to do a second release, please remove the tag and add it to the new version of the source file. If you do not know how to do that, check the cvs documentation.
    If you still do not know how to do it or for some reason want to keep the release that was made earlier, make a tag "rel_YYMMDD_X" where X is starting with 2, e.g. "rel_021229_2".

    Further on, after you tagged your files we need to map these files to a specific binary. Therefore we put the date of the release in the binary name with the pattern "binary.YYMMDD", e.g. "calculateRevenue.021229".

    Ok, how does this work in practice?
    Since we are using makefiles for building our binaries, it would be convenient if we could do a "make install" to install the binary, and that is exactly what we will do.
    An example will probably make it clear.

    First we will create our binary, to make sure it is all nice and clean we do a make clean first. 

     user12@calvin% make clean
         rm -f Revenue_main.o Stuff.o OtherStuff.o FileHandling.o Calculation.o
               RevenueCalc.o RevenueCalcerrors.o calculateRevenue
 user12@calvin% make
         aCC -c  -I. -I/opt/aCC/include -I/opt/aCC/include/SC ./Revenue_main.cc -o ./Revenue_main.o
         aCC -c  -I. -I/opt/aCC/include -I/opt/aCC/include/SC ./Stuff.cc -o ./Stuff.o
         aCC -c  -I. -I/opt/aCC/include -I/opt/aCC/include/SC ./OtherStuff.cc -o ./OtherStuff.o
         aCC -c  -I. -I/opt/aCC/include -I/opt/aCC/include/SC ./FileHandling.cc -o ./FileHandling.o
         aCC -c  -I. -I/opt/aCC/include -I/opt/aCC/include/SC ./Calculation.cc -o ./Calculation.o
         aCC -c  -I. -I/opt/aCC/include -I/opt/aCC/include/SC ./RevenueCalc.cc -o  ./RevenueCalc.o
         aCC -c  -I. -I/opt/aCC/include -I/opt/aCC/include/SC ./RevenueCalcerrors.cc  -o ./RevenueCalcerrors.o
         aCC -o calculateRevenue Revenue_main.o Stuff.o OtherStuff.o FileHandling.o Calculation.o
                RevenueCalc.o RevenueCalcerrors.o -L/usr/lib/ -lrwtool -lcl -ldce -lcma
    We have now created the binary "calculateRevenue". To install this binary we will do "make install" with an argument MSG="<comment why this binary is released>" 
     user12@calvin% make install MSG="improved conversion functions"
         install.ksh -b -m 'improved conversion functions' /home/user12/theFiles/project1/calculation/"calculateRevenue.`date +%y%m%d`"
 ----------------------------------------------------------------------
 Logging user as user12
 Comment: improved conversion functions
 FTPing
   calvin:/home/user12/theFiles/project1/calculation/calculateRevenue.030306
 to
   hobbes:/opt/calc_data/
 ---------------------------------------------------------------------
 Enter password for user prod87 :
 
 user12@calvin%
    This will first copy your binary to todays date, in this case calculateRevenue.030306, and then transfer the binary to the production server hobbes.
    It will also create a symlink on the production server calculateRevenue -> calculateRevenue.030306,
    Hence we can execute calculateRevenue, but we now have knowledge which binary that is active, and which set of source code that we used to build this binary (all source code that is tagged with rel_030306 in this case).
    All the old binaries are remained on the server, so if the new binary seems to be faulty for some unknown reason, you only have to change the symlink to point to the second newest binary.

    After the binary is installed you should log on to the production server and do a test run to see that the binary is working, note that you should use the symlink calculateRevenue rather than the full name of the binary (so you know that the symlink is working fine)

    More programming related pages

    Workflow: Release process
    Workflow: Bug tracking
    Teambox (Redbooth) - Mantis connector
    Design Patterns
    Git & Github
    		Go / Golang
    CVS
    CVS backup script
    Distribution process
    Installation script
    		Java Server Faces
    Facelets
    jibx
    jBoss
    jBpm
    		Perl tips
    Perl links
    PostgreSQL
    Python / py2app
    Shell scripts
    		Xslt
    Node.js
    Facebook / Opengraph
    PHP developer notes
    Redbooth API through php
    		Website optimization
    jqTableKit demo
    Javascript / html / css links
    		•