# Copyright (c) 1997 by Jim Lynch.
# This software comes with NO WARRANTY WHATSOEVER.
#
# This program is free software; you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation; version 2 dated June, 1991, or, at your
#    option, any LATER version.
# 
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
# 
#    You should have received a copy of the GNU General Public License
#    along with this program;  if not, write to the Free Software
#    Foundation, Inc., 675 Mass Ave., Cambridge, MA 02139, USA.
# 
# On Debian Linux systems, the complete text of the GNU General
# Public License can be found in `/usr/doc/copyright/GPL'.

CREATING THE SCRIPT WHICH PROCESSES ROSTER DATA IN YOUR SCHOOL'S FORMAT
-------- --- ------ ----- --------- ------ ---- -- ---- -------- ------

Notes: You need to create a perl script that converts the student data
you receive from your equivalent of Admissions and Records, to my roster
file format. This is (presently) a text file whose lines represent one
student who is enrolled in one class, and includes the student's ID code.

To form the record in the proper format, your script should pick up the
following data, regarding a single student in a single class: for this 
example, you would have some method of creating a login name for the 
student (first five letters of last name followed by first initial then
middle initial has been known to work) allowing for one or two more positions
in the name for extra digits in case two or more students have the same
login name. The script should form this login name and place it (for this
example) in $logname.

The variable $name should receive the raw full name, totally unmodified,
from the download, spaces and all.

You should have a unique identifying number (or other tag) for the specific
course during a specific semester. Your school is probably larger, but ours
only needed a four-digit number to identify a given course unique within
a semester. This code (which should appear in the school data) should go in
the variable $classCode.

Next, the code or number which uniquely identifies a student would go in 
$studentID. If you are in a multicampus environment and the student IDs
are not unique across campuses, you might add the campus code to the front.
Note, however, that it might be better to have the IDs apply to the student
on all campuses.

Our district has 4 campuses, and it turns out that they all start with a 
different letter, so we used that letter to identify the campus. You should
put such an identifier in $campus. I have to look at the code I wrote to
remember exactly what I do with it; I think nothing since I have not yet
considered implications of multicampus operation beyond the use of the
class code and the campus ID together to uniquely identify any course in
a district of campuses.

The semester (aka term elsewhere in my code) should be placed in $semester.
Note that this value is used in many places to make sure databases for 
differing semesters don't mingle. (Note: the value that goes here also
would go into a file in the data directory to cause all the scripts to 
operate on the files relevent to that term.)

See if you can get the admin of your school to have the records given to 
you even if the student is dropped or on a waiting list. Ask to have
2 extra fields: one being some indication of whether or not the student
has dropped the course, and the second being an indication of whether
or not the student tried to enroll but was placed instead on a waiting
list. If there are other possible indications of status for a student,
ask to be supplied with information regarding that. The idea being:
the variable called "$status" needs to contain the status of the student,
either the student is "current", "dropped" or on the "waitlist".

If it is not possible to get these fields, then ask for records containing
-only- currently enrolled students.

Note: if it turns out that the only way of knowing if a student is dropped
is by way of a record being missing in the download, then you may have
to keep a separate database in order to tell if you have seen the student
in that class as indicated in a previous roster download, and hence to be 
able to conclude that the student's status is indeed "dropped".

Once you know the student's status, put this in a variable called $status.

Once all this data has been collected, you can form a data record from it
by calling the following:

        $outLine = PackRostRecordFromHashRef
        ({
            InitLogName =>  $logname,
            FullName    =>  $name,
            ClassCode   =>  $classCode,
            StudentID   =>  $studentID,
            Campus      =>  $campus,
            Semester    =>  $semester,
	    Status      =>  $status
        });

The data record will then be written to the variable $outLine. Once you have
the value there, you can send that line to the file whose name is in the
variable $DataIndepRostFile. The definition of this variable is in paths.perl.

The name of the file you read (the download from Admissions and Records) is
in $DataRawRostFile, and you should edit paths.perl to change the name of
this file to the name you would like this file to have. In my experience, 
having the file name be the same name as the downloaded file is an easy way.

Overall, your script should have these lines at the top of the file:


---- cut here
#!/path/to/your/perl


use FindBin; use lib "$FindBin::Bin";

require 'functions.perl';

GetTerm();

---- cut here


You should then open the downloaded file ($DataRawRostFile) for reading, and 
the roster file ($DataIndepRostFile) for writing.

Next, you sequence through the raw roster records (which are hopefully on
individual lines). 

For each raw roster record,
  first check the supposed roster record to determine if it's a likely record
  (I have done this on occasion by checking the number of fields on the line)
  and if it looks like a raw roster record, 
    gather necessary data
    do any necessary conversions (including synthesizing the login name)
    feed this properly converted data to PackRostRecordFromHashRef
    append it to the independant-format file

Lastly, you would close the raw roster file, and close the independant-format
roster file.

Once you run this script on your roster data, you will have created a file
in the format used by my scripts.

I will probably make a few example scripts for ficticious
roster formats. I might use real formats given releases from the chancellor(s)
of the multicampus district in question; I am unlikely to go through this
hassle, however! The release would have to state that the data format is copy-
right by whoever, has no warranty whatsoever, and is released under the 
complete control of the GPL, with absolutely no exception, addition or
exclusion whatsoever. Also, an agreement would have to be made that 
the placement of the data format in question under the terms of the GPL
would never, ever be revoked at all for any reason whatsoever. Then I would
consider making that data format an example here.

KNOWN BUGS
----- ----

There is presently no handling of exceptional cases. This is an area I would
welcome some help on. It makes this software "beta" at best. One case in 
particular that came up recently: the software does not perform well at all
in the absense of disk space!!

This release is work-in-progress; I'm in the middle of fixing a bug in which
the rosters for the classes seem to grow and grow... seems I wasn't 
interpreting the data properly: I looked at the records I already had for
a student, and the records that came from the latest download, and simply
"or"ing together the classes, when in fact I should be mirroring the
Admissions and Records data, dropping students from classes, and finally
disabling accounts when A&R says the student is in no classes.

EVERY FEW SEMESTERS
----- --- ---------

There are approximately 24,000 user IDs available in the system.
Every once in a few semesters, you will run out of IDs. At this 
point, it would be a good idea to get all the users.db files
for semesters between the times you run out of IDs and have to
flush the system out, and recreate the lists of available user IDs.

Running any of this for the first time ever MAY REQUIRE FULL BACKUPS
and MAY ALSO REQUIRE !!ALL!! NON-SYSTEM ACCOUNTS TO BE DELETED.

If you are running all of this for the first time ever, start with
a _fresh_ box, i.e., root and system accounts only.

Then run: create-free-id-databases.perl.

The idea of this, is to have _all_ accounts have the same unix user
ID on all unix machines. This will allow simple migration to a distributed
unix platform.

What you will need to do for students, is DELETE ALL THE STUDENT 
ACCOUNTS and run a perl script containing the following code:

--- cut here ---
#!/path/to/perl

# create databases containing unallocated unix user IDs, in the
# category student.
#
# Copyright (c) 1997 by Jim Lynch.
# This software comes with NO WARRANTY WHATSOEVER.
#
# This program is free software; you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation; version 2 dated June, 1991.
# 
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
# 
#    You should have received a copy of the GNU General Public License
#    along with this program;  if not, write to the Free Software
#    Foundation, Inc., 675 Mass Ave., Cambridge, MA 02139, USA.
# 
# On Debian Linux systems, the complete text of the GNU General
# Public License can be found in `/usr/doc/copyright/GPL'.

require 'functions.perl';

$debug = 0;

$timeOfThisRun = time;

CreateAvailUnixUserNumberDB("student");

rename("unixids.txt", "free.student.ids.txt");

--- cut here ---

RUNNING FOR THE FIRST TIME IN A SEMESTER
------- --- --- ----- ---- -- - --------

One thing that you might find important to do, is to back up the
UNIX home, mail and web page directories together with their 
corresponding users.db database. I leave it to you to decide
if and how to do this.

You need to create a file called term.txt in order to identify
which semester you are creating logins for. This file will inform
the entire database system of the term, and cause the system to name 
files appropriately. 

The file must have exactly one line with no spaces. All this really
does is cause files written by the system to contain the term in them.
For example, if it's Summer of '97, and you therefore put M97 (because 
S would be Spring) into term.txt, then M97 will be inserted into the
names of all (most?) of the files the system writes. So, if the file is 
referred to here as codegroup.colon, its name will be
codegroupM97.colon.

 is the identifier for the semester, and should be a letter
followed by the year. The letter would be F for fall, S for spring
and M for summer. So, for Summer '97, the  would be M97, and
you would therefore put M97 into term.txt.


Next, you must create a database associating each class code with a
corresponding group name. The preferred method for doing this, is to
simply create codegroup.colon in a plain text editor, such as 
emacs. (this will change, but for now is _preferred_.) Each line is 
of the form:

classcode:groupname

which will cause an association between the class codes seen coming
from the district computer, and a selected group used internally
by a computer lab.

You should take a look at the sample codegroup.colon file to see
what's being asked for. As an aside, if you do type in the file
yourself, it will be unnecessary to use foxpro for the purpose of 
getting the class codes associated with certain groups. 


Alternatively, you can go to foxpro with the classes.dbf file and save 
this file in foxpro as a tab-separated database; Use the name classes.tab 
for the output. Transfer the file using FTP in ascii transfer mode, as above,
to the directory you're working in.

Following this, you take the classes.tab file and process it through
classes.tab2codegroup.colon.perl, whose output should be fed to 
sort -u. This will sort the file and get rid of duplicates.
The output will be stored in a file called codegroup.colon.


Note: before you ask your novell server to create the accounts, you 
need to make sure the groups exist on that server. The same is true for
the unix accounts: before using newusers, the groups must exist.

If you are running the database for the first time in a semester, you can
get rid of the old users.db file. All you would need to do is delete 
(or move) the file. The important thing, is that term.txt is correct (or at
least unique) for the semester.


MERGING NEW STUDENTS INTO THE DATABASE AND CREATING LOGINS
------- --- -------- ---- --- -------- --- -------- ------

These steps are to be done:
  1) when creating the database for the first time in a semester
     (!AFTER! performing the above steps); and
  2) when merging students who add classes after you create
     the database for the first time.

The first step is to use FTP in ascii transfer mode to get the 
cis_rost.txt file from the district to the dir that has all the
programs in it (this will change, but for now, dump it in with the 
scripts).

Note: Using the correct FTP transfer mode is very important! 
Actually, it's desirable for my program to overlook a mistake like
this. For now, however, you must use FTP in ascii mode.


Once you have the latest cis_rost.txt file from the district, you 
need to create the simpler database format. This is done by using 
cis_rost.txt2cis_rost.db.perl. At this point, you should have 
cis_rost.txt and cis_rost.db in the same dir with all the scripts. 
Take a quick look at both files: do they seem reasonable?

users.db is the _main_ database; it contains records for _all_
the students in _all_ classes mentioned in codegroup.colon.

unix.txt and users.usr are to be considered TEMPORARY files.
They will very likely shrink. If you are looking for passwords or
student ID codes, look in the main database.

Once you have done this, the next step is to merge cis_rost.db 
in with users.db. To do this, run merge-cis_rost.db-users.db.perl. 
This will cause users.db to grow.

To create the main database, run merge-cis_rost.db-users.db.perl.
This will produce users.db.

cis_rost.txt is now completely processed; form a date as follows:
The date should be of the form YYYYMMDD, so Thu Jan  8 1998 is 19980108.
Next move the used cis_rost.txt to old-rosters/cis_rost.txt-date where
date is today's date in YYYYMMDD form.

To create the users.usr and unix.txt files, simply run 
mk-logins.perl. Doing so will write NEW versions of users.usr
and unix.txt. NOTE: other files are produced for the unix side!
Make note of what it says!

META: either clean this up or make the program work with one file

You can then create and/or update novell logins by first FTPing the 
users.usr file to a novell workstation (as usual, in ascii
transfer mode), add commands at the top of the file as appropriate
to your login defaults preferences and submit the combined file to
novell's MAKEUSER.

You can also create and/or update unix logins by transfering the
unix.txt file to the unix machine(s) and running:

newusers unix.txt

and you will get the new users. Note: All the unix users will have
the same user ID number on _all_ unix systems you add them to. You
!!MUST!! make sure the IDs are available on _all_ machines!