---------------------------------------------------------------------------------
    SWITCHING THE CODA DATABASE FROM MSQL TO JDBC DATABASE
---------------------------------------------------------------------------------

CodaDatabaseProxyServer.java is the source code for a server that translates the
original libmsql.c communication with an msql database server, to communication
with a jdbc database. This is done in conjunction with a new msql library called
libmsqlJDBC.c. It is designed to have all CODA database applications work
seamlessly with a new database.

There are several steps necessary to carry this out properly:

    1) Installation of a new database
    2) Installation of the java JDBC database driver
    3) Modifying CODA's msql distribution & recompiling
    4) Running the proxy server, and
    5) Transfering tables and data from msql to the new database


INSTALLATION OF A NEW DATABASE

    Since this step is highly dependent on which new database is being used,
    the following will run through the step for installing MySQL.
    
    - Go to www.mysql.com
    
    - Go to "production" under the "Database Server" area
    
    - Download and install both the server rpm (eg for Redhat Linux)
      and the client programs rpm
      
    - Read the online manuals for detailed information on how to
      start the mysql server. But in a nutshell, first run the
      setup script "mysql_install_db", and then run the server,
      "mysqld_safe --user=mysql &" 
      .
    - Make sure all installation file permissions are properly set. This means
      make sure all files in /var/lib/msql are owned by the user "mysql" and
      the group "mysql"
      
    - By default the anonymous user has no priviledges. This can be changed in
      the following way:
      
      Download the "Control Center" gui software from the same site and run it.
      Using this gui, look at the "user" table of the "mysql" database. Give
      select, insert, update, delete, create, and drop priviledges to the anonymous
      use or the user of one's choosing. Do so by changing the data in the 
      appropriate columns from "N' to "Y"
      
    - Check to see if the server is running by running something like
      "mysqladmin version". This will print out version information etc.
      
      
INSTALLATION OF THE JAVA JDBC DATABASE DRIVER

    Since this step is highly dependent on which new database is being used,
    the following will run through the step for MySQL.
    
    Make sure that the JDBC driver class file is in the classpath when the
    proxy server is run. For MySQL, you can download the java JDBC driver
    called "Connector/J" from www.mysql.com. Put the driver in some convenient
    place and be sure to include that path in your CLASSPATH environmental variable.


MODIFYING CODA'S MSQL DISTRIBUTION
    
    Directory Assistance:
    
    Go to the "msql/targets" directory of the CODA source distribution.
    Go to the specific target subdirectory of the operating system of interest
    (eg. "Linux-2.4.7-10smp-i686" directory of Redhat 7.3 Linux system).
    Go to the msql subdirectory.
    
    
    Changing Makefiles:
    
    In "Makefile.full" have the programs "msqldump", "msql", and "msqld"
    link against libmsqlJDBC.a NOT libmsql.a.  Do NOT change msqld
    UNLESS you intend to have clients talk to the proxy server.  
    
    Recompile and reinstall by doing a "make install" either in that directory
    or in the one above. Many other parts of CODA which depend on libmsql.a must
    also be recompiled and reinstalled against libmsqlJDBC.a  This is best done
    by renaming libmsqlJDBC.a to libmsql.a in the CODA binary distribution 
    (eg. $CODA/Linux/lib) and then recompiling the following CODA directories:
    cedit, dac, dp, dp/dplite, and tcl_msql.
    

    OPTIONAL - Making shared libraries
    
    If the user wants to flip-flop between using the java proxy server and talking
    directly to the msql database server, it is easiest to make 2 shared libraries
    and switch between them instead of recompiling and reinstalling parts of CODA.
    
    The ROC code which runs on vxWorks will NOT be able to use shared libraries to
    compile so don't bother to do the following for the msql/targets/<vxWorks system>
    Makefile.full.
    
    One way to do this is to go to the msql/targets/<operating system> directory.
    From there, first go to the "common" subdirectory and modify "Makefile.full"
    so that "CFLAGS" has the flag "-fPIC" added for Linux or "-KPIC" added for
    Solaris. Type "make" to recompile.
    
    Next, go up one directory to msql/targets/<operating system> and then to the
    "msql" subdirectory. Again, modify "Makefile.full" so that "CFLAGS" has the
    flag "-fPIC" added for Linux or "-KPIC" added for Solaris.
    
    Add the following lines to the appropriate parts of "Makefile.full"
    

all : libmsqlJDBC.so

libmsqlJDBC.so : ../common/strlib.o ../common/tmpnam.o libmsqlJDBC.o net.o
	ld -G ../common/strlib.o ../common/tmpnam.o libmsqlJDBC.o net.o -o libmsqlJDBC.so

clean ::
	rm -f libmsqlJDBC.so


all : libmsql.so

libmsql.so : ../common/strlib.o ../common/tmpnam.o libmsql.o net.o
	ld -G ../common/strlib.o ../common/tmpnam.o libmsql.o net.o -o libmsql.so

clean ::
	rm -f libmsql.so

    
    This tells "make' how to make the shared libraries and will work for both Linux
    and Solaris. If you want to be able to type "make install" and have the shared
    libs installed, then add the following lines to the appropriate part of
    Makefile.full :
    
    
    Now, change the lines:
    
msql : msql.o libmsql.a
	$(LINK) $(CC_FLAGS) msql.o libmsql.a -o msql $(LD_LIBS) 
    
    To:

msql : msql.o libmsqlJDBC.so libmsql.so
	$(LINK) $(CC_FLAGS) msql.o -lmsql -o msql $(LD_LIBS) 
    
    
    This tells msql to link against the shared library instead of the local
    static library. Just make sure your LD_LIBRARY_PATH does not have "./" in
    it before $CODA/Linux/lib.
    
   
install ::
	if test -f $(CODA)/$(OSTYPE)/lib/libmsql.so;\
	then\
		rm -f $(CODA)/$(OSTYPE)/lib/libmsql.so.old;\
		mv $(CODA)/$(OSTYPE)/lib/libmsql.so $(CODA)/$(OSTYPE)/lib/libmsql.so.old;\
	fi;\
	cp libmsql.so $(CODA)/$(OSTYPE)/lib/libmsql.so
	chmod 744 $(CODA)/$(OSTYPE)/lib/libmsql.so

install ::
	if test -f $(CODA)/$(OSTYPE)/lib/libmsqlJDBC.so;\
	then\
		rm -f $(CODA)/$(OSTYPE)/lib/libmsqlJDBC.so.old;\
		mv $(CODA)/$(OSTYPE)/lib/libmsqlJDBC.so $(CODA)/$(OSTYPE)/lib/libmsqlJDBC.so.old;\
	fi;\
	cp libmsqlJDBC.so $(CODA)/$(OSTYPE)/lib/libmsqlJDBC.so
	chmod 744 $(CODA)/$(OSTYPE)/lib/libmsqlJDBC.so


    So, after making all these changes, do your "make install". Then go to your CODA
    binary installation in the lib subdirectory (eg $CODA/Linux/lib), get rid of or
    rename the "libmsql.a" and "libmsqlJDBC.a" files. Recompile the CODA directories:
    rc, cedit, dac, dp, dp/dplite and tcl_msql. Now everything should be linked against
    the shared libraries.
    
    ****************************************************************************
    * To use the proxy server, change the name of libmsqlJDBC.so to libmsql.so *
    * in the CODA binary distribution.                                         *
    ****************************************************************************
    
    Initially to talk to the msql server directly (as was the standard practice
    in the past), nothing needs to be done. Once, however, the original
    libmsqlJDBC.so was renamed to libmsql.so, to switch back simply rename what was
    the original libmsql.so back to its original name (libmsql.so).

    If all of this confuses you, seek professional help.
    

RUNNING THE PROXY SERVER

    Running the JDBC server:
    
        Make sure that the JDBC database server is running. This is highly
        database dependent, but in the case of MySQL it can be run with
        "mysqld_safe --user=mysql &" .
        
        Of course, it is also possible for the JDBC server to be msql since the
        msql server has a JDBC interface. In this case, execute "msqld&"
        as before. BUT, MAKE SURE "msqld" IS COMPILED AGAINST THE OLD OR ORIGINAL
        libmsql.a OR libmsql.so.
        
        When the new "msqld" starts up, it first checks the environmental variable
        CODA_DB_PORT so see which port to listen on. If that is not defined it
        checks MSQL_TCP_PORT. If that is not defined it looks to see if the local
        host has defined an "msql" service in the "/etc/services" file amd uses that
        port. If that fails, it uses 8101.
        
        So if the msql server and the proxy server are going to be running on the same
        host, the proxy server needs to be listening on MSQL_TCP_PORT, so the msql
        server needs to use a different port. Define it with CODA_DB_PORT.
        
         
    Running proxy server:
    
        The CODA source distribution keeps the code for the proxy server in the
        <main distribution dir>/msql_tools directory. Class files generated by
        compiling are installed in $CODA/common/lib/java. These are:

            CodaDatabaseProxyServer.class
            UnixEnviroment.class
            ClientThread.class

        First, some environmental variables need to be set:

            CODA_DB_PORT
            CODA_DB_DRIVER
            CODA_DB_URL
            CODA_DB_USER
            CODA_DB_PASSWORD
            MSQL_TCP_PORT
            MSQL_TCP_HOST
        
        CODA programs that currently talk to the msql server use the env variable
        MSQL_TCP_HOST to tell which host it is on. Set this to point to the host
        that the proxy server will run on.
        
        The proxy server listens on MSQL_TCP_PORT. If that is not defined it listens
        on 8101.
        
        It looks at CODA_DB_DRIVER to get the name of the JDBC driver java class.
        For MySQL this will probably be "com.mysql.jdbc.Driver". For msql (at 
        least in the version 1 that CODA runs) it is "com.imaginary.sql.msql.MsqlDriver".
        
        It looks at CODA_DB_URL to get the host, port, and database name of the
        JDBC data server. For MySQL this will be something like:
        "jdbc:mysql://myhost.jlab.org:3306/my_database". Each user will have to 
        substitute his own host and database names.  For msql it will be like:
        "jdbc:msql://myhost.jlab.org:8101/my_database". Obviously the host and port
        and database name will be different for msql.
        
        The proxy server looks at CODA_DB_USER to get the name of the database user
        account. MySQL can be setup for an anonymous user in which case this 
        environmental variable does not need to be defined or can be blank.
        Msql does nothing with the user name.
        
        Finally, the server looks at CODA_DB_PASSWORD to get user's password. MySQL can
        setup user accounts with or without passwords. If no password is needed, this
        env variable can be left undefined or blank.
        Msql does nothing with the user password.
        
        Having said that, these environmental variables may be usurped by specifying
        values on the command line. The following shows how to override the env
        variables:
        
        java -Ddriver=blah -Duser=blah -Durl=blah -Dpassword=blah ...
        
        Now we have come to the point of showing how to run the server. Simply type:
        
        *****************************
        java CodaDatabaseProxyServer
        *****************************        
        
        It may be a bit more useful to specify a classpath which includes CODA stuff.
        The following CLASSPATH picks up the proxy server's class files as well as the
        jar file containing the MySQL java driver:
        
        setenv CODA_STUFF $CODA/common/lib/java/mysql-connector-java-3.0.8-stable-bin.jar:$CODA/common/lib/java
        java -cp $CODA_STUFF/:$CLASSPATH CodaDatabaseProxyServer
        
        At this point you can test things out by running a program like dbedit.
        However, if there is no data being server by the new JDBC server, there is
        no point in doing so. First follow the directions in the next section to
        transfer old data from msql to the new database


TRANSFERRING TABLES AND DATA FROM MSQL TO THE NEW DATABASE

    1. Save your msql database to a file with "msqldump -c <databasename> > <filename>"
       (eg. msqldump -c carls_db > mydumpfile.txt).
       
    2. At this point some editing of the dump file may be necessary - it is if it's
       to be used with MySQL. The trouble is that character fields in a table may be
       quite large in msql but are limited to 255 characters in MySQL. In CODA, the
       main table associated with a particular configuration has character fields of
       both 512 and 400 characters in length. Since the dump file (mydumpfile.txt in
       the above example) is ascii, it's easy to replace all occurrances of 
       "CHAR(512)" and "CHAR(400)" with "CHAR(255)".
       
    3. Create a new database in your JDBC server into which you'll put the dumped
       data (eg. carls_db2)
       
    4. After making sure that "msql" was linked with the new libmsql, use 
       "msql <newdatabasename> < <filename>" (eg. msql carls_db2 < mydumpfile.txt).
        
    
    Using some sort of database browsing tools such as CODA's "dbedit" or MySQL's
    "mysqlcc", you can browse the new JDBC and confirm that all the data was
    transfered as expected.
    
    
---------------------------------------------------------------------------------
    USING DATABASE-EDITING PROGRAMS
---------------------------------------------------------------------------------

    CODA provides "dbedit" to edit msql databases. It is written in tcl/tk with an
    extension which interfaces it to the msql version 1 database.
    
    Since msql and many other databases have java jdbc drivers available, any java-
    based database editor should work. In CODA's msql_tools directory, jar files for
    jdbc drivers for both msql and MySQL are included. These java-based editors
    will work with the msql database only if it's compiled against libmsql and not
    libmsqlJDBC.
    
    
    

NOTE:
  Java jar files must be check into cvs with the command:
    cvs add -kb file.jar
  The kb option treats it as a binary file. Without this option,
  the jar file gets corrupted.
  
  When checking out a jar file, the same flag must be present, for example.
    cvs update -kb file.jar

     

For any questions contact Carl Timmer, timmer@jlab.org
