File Archiving - Collection Management and Organization

I am a collector! More correct would be to say that I hate to throw stuff and that I try to accumulate any data I can get my hands on about any subject of great interest to me.

This must be a trait that I inherited from my dad, who drives my mom nuts with his collector habits.

My Dad's Collecting Habit

The problem that my dad has is the fact that he collects only physical stuff, such as Coins (legal tender coins only), Stamps (West German and American only), Post cards from cities and towns with crest of that town on it (a shi*tload of towns in Europe have their own crest and are only happy to print them on post cards of their city, much to the dismay of my mother), post cards with only a single picture, which must be a true aerial photograph (no picture from a tall building, mountain or anything like that), Locomotives and box cars in model format, manufactured by the former East German company "TT-Eisenbahnen" (shut down a couple years after the wall fell, but still alive and supported by fans and model train owners).

I think that is it... I hope that I did not miss any. As you can see, always very specific subjects, but then more than one subject, which created a slight inventory problem for my parents, which was only resolved when my sister and me moved out into our own places as soon as possible (I moved out with 19, but my parents paid for the basic rent hehe).

My Own Collection Habits from the Past

I used to collect stamps as a kid (only "Space" and "Astronomy" motives) and for a brief period of time "crown caps" where the product or company name was printed on the metal cap. This was rarely done in East Germany, but typical in the Western countries. I abandoned this collection around the time when the wall came down. Go figure. The only physical thing that I collected since 1982 until today are the print issues of the oldest and most popular (of only 3 or 4 or so) East German comic magazine called "Mosaik", which was published monthly since December 1955.

The other physical stuff is not really collecting, because if there is not much to collect, then I don't call it a real collection, even if I have everything (or most) of it. Into this category would be my Cirque du Soleil DVDs and CDs and my VNV Nation stuff and support as a fan of the band.

Digital Collections

However, my collector habit manifested itself in its full extend on the computer where the stuff that you collect is purely digital in nature and thus only uses very little storage space (compared to physical collections). It started when I did the Warez stuff and even more so, when I ran my own BBS. I automatically came across (and got a copy) the most recent software that mattered in those days (worthless junk remains worthless junk and is not worth stashing  up anywhere, even if it is digital junk). I never threw away or deleted any software that I had a copy (or original of), even if the software was too old for "trading" in Warez boards and I also never used the software myself ever. I had it, that was all that mattered.

My software collection was unfortunately destroyed by the German Police, who took away any data storage media that was not an original and they could find during the raid of my apartment (because of my BBS "Closed Society") back in spring 1997. That was the part that hurt me the most... they destroyed everything... it was not just stored somewhere else to still exist as a whole, but inaccessible to me.

Anyhow, things changed with the Internet. A lot of old and forgotten stuff by mainstream culture can now be found online. Not only software like games, but other things that were created as well. I was interested in computer art for example, specifically ANSI and ASCII text art and pixel art. I created stuff myself. I was also interested in the demoscene, that produced and still produces tons of new demos, created by folks like me for the pure sake of creating them and showing them off.

The copyright situation for most of this old stuff is unclear, not specified or simply not enforced by anybody who could make claims of the intellectual property created just for fun, without commercial background. In other cases is the content still officially "copyright protected", but the owner is long out of business or lost any interest in his own property, because he thinks that it does not have any commercial value anymore. I wrote about those things in the past already.

I have a bunch of collections of various kinds. Art, Pixel Art Fonts, Pixel Art Logos, Music, Software.... or using a  simple word that by the end of the day categorizes them all... ...Files.

I am not as bad as Jason Scott from textfiles.com who's collecting habits and urges dwarf mine in comparison.

Keeping Stuff Organized

Organizing these vast amount of items, which are in most cases ridiculous small in size for today's standards, is a pain in the neck and time consuming. I frequently stumble across collections created by others to the same subject that is of interest to me and sorting stuff out, is usually not that easy. Most of those collections are simply a dump of hundreds or thousands of files, each item packed with ZIP, RAR, LHA etc. and then packed together ones more to a multi-part RAR archive or ISO DVD/CD image.

The stuff I usually do with each of those "dumps" is the same, so I wrote myself some small scripts to make things easier for me and to speed up the archiving and sorting process.

The collection that I download are usually just dumps of many files all lumped into a single directory ones you unpacked the RAR archive or ISO image. I always break those up and sort them into individual sub directories by the first letter of the title of each item. The file names are in almost every case the title of the software, demo production, image, mod file, PDF document, text file etc. So I always have to create 27 directories (A-Z plus one for all titles that start with a number, titled 0-9) and then move the files to their respected directories.

An index of all the files is usually also missing to my script is generating one for me. A clean and simple index, which you could not achieve without manually tweaking it, with the standard DIR command in MS DOS.

My Free Helper Scripts

You can download the source scripts packed into a single ZIP archive: !archive_scripts_src_roysac.zip

This batch file creates sub directories "0-9", "A", "B" ... "Z", If they do not already exist (script: !folders.vbs) and then moves all files in the current directory to their corresponding sub folders (part of batch: !organize.bat). After the files were moved, an index file is created (script "!dirlist.vbs") with the name "!" + the current folder name + ".txt". The index file lists all files in the previously created directories. It ignores any other sub folder that might exists in the same dir.

All this is done by three scripts. You only have to run the main one. It calls the other two when needed. The script does not have or require any parameters or configuration options. Just copy the three files into the folder where you dumped all the files and run "!organize.bat".

!organize.bat Script

@echo off

REM      !organize.bat 

REM      Batch File by Carsten Cumbrowski, March 2009

REM       -----------------------------------------------------------------------------------

REM      Required support files used by this script:

REM       - !folders.vbs

REM       - !dirlist.vbs

REM      PURPOSE

REM       -----------------------------------------------------------------------------------

REM      This batch file creates sub directories "0-9", "A", "B" ... "Z"

REM      If they do not already exist and then copies all files in the 

REM      current directory to their corresponding sub folders

REM      (Yeah, that's why all of the script files start with a "!" in the file name)

REM      After the files were moved, an index file is created

REM      with the name "!" + the current folder name + ".txt"

REM      The index file lists all files in the previously created directories

REM      Note: the script ignores other sub directories that might

REM      exist in the current folder, also files that don't start with

REM      0-9 or A-Z (a-z) in the name are ignored

REM       -----------------------------------------------------------------------------------

REM      Copyright: NONE, Use it! Improve it! Share it!

cls

REM -----------------------------------------------------------------------------

REM Create Sub Directories 0-9, A, B ... Z If they not exist

REM -----------------------------------------------------------------------------

echo Creating Sub Directories 0-9, A, B ... Z ...

wscript !folders.vbs

REM -------------------------------------------------------------------

REM Move files into their respective sub directories

REM -------------------------------------------------------------------

echo Moving Files to Sub Directories ...

move 0*.* 0-9\

move 1*.* 0-9\

move 2*.* 0-9\

move 3*.* 0-9\

move 4*.* 0-9\

move 5*.* 0-9\

move 6*.* 0-9\

move 7*.* 0-9\

move 8*.* 0-9\

move 9*.* 0-9\

move a*.* a\

move b*.* b\

move c*.* c\

move d*.* d\

move e*.* e\

move f*.* f\

move g*.* g\

move h*.* h\

move i*.* i\

move j*.* j\

move k*.* k\

move l*.* l\

move m*.* m\

move n*.* n\

move o*.* o\

move p*.* p\

move q*.* q\

move r*.* r\

move s*.* s\

move t*.* t\

move u*.* u\

move v*.* v\

move w*.* w\

move x*.* x\

move y*.* y\

move z*.* z\

REM -------------------------------------------------------------------

REM Create !<FolderName>.txt Index File 

REM -------------------------------------------------------------------

echo Create !<FolderName>.txt Index File ...

wscript !dirlist.vbs

REM -------------------------------------------------------------------

REM Done!

REM -------------------------------------------------------------------

echo done! 

echo.

pause

!folders.vbs Script

'---------------------------------------------------------------

'Script Name: !folders.vbs

'---------------------------------------------------------------

'VBScript by Carsten Cumbrowski, written in March 2009

'---------------------------------------------------------------

'Purpose: Creates Sub Folders  "0-9", "A","B" ... "Z" 

'         in the current folder, if they do not exist already

'         The Script was created for the use with collection

'         of scripts and batch files for archiving and fileing

'         of software and file collections.

'---------------------------------------------------------------

'Parameters: No Parameters Required or Supported

'            Simply run the script from the DOS command prompt

'            in Windows by typing "wscript !folders.vbs" 

'---------------------------------------------------------------

'Copyright: NONE, F**k that shit! Use it! Improve it! Share it!

'---------------------------------------------------------------

Dim oFso, a

Set oFso = CreateObject("Scripting.FileSystemObject")

if NOT oFso.FolderExists("0-9") then

  oFso.CreateFolder("0-9")

end if

For a = 65 to 90

  if NOT oFso.FolderExists(chr(a)) then

    oFso.CreateFolder(chr(a))

  end if

Next

Set oFso = Nothing

!dirlist.vbs Script

'---------------------------------------------------------------

'Script Name: !dirlist.vbs

'---------------------------------------------------------------

'VBScript by Carsten Cumbrowski, written in March 2009

'---------------------------------------------------------------

'Purpose: Generates a .txt file with the name "!" + name of 

'         current folder. The text file contains a list of all

'         files in the sub directories with the names "0-9",

'         "A","B" ... "Z" and totals for the number of files

'         and size in bytes of each sub folder and in total.

'         The Script was created for the use with collection

'         of scripts and batch files for archiving and fileing

'         of software and file collections.

'---------------------------------------------------------------

'Parameters: No Parameters Required or Supported

'            Simply run the script from the DOS command prompt

'            in Windows by typing "wscript !dirlist.vbs" 

'---------------------------------------------------------------

'Copyright: NONE, F**k that shit! Use it! Improve it! Share it!

'---------------------------------------------------------------

'Declaration of Constants

Const ForAppending = 8

Const SeparatorLength = 70

'Declaration and Initialization of Work Variables and Objects at Runtime

Dim oFso: Set oFso= Wscript.createobject("scripting.fileSystemObject")

Dim sFolderPath: sFolderPath = oFso.getAbsolutePathName("")

Dim oFolder: Set oFolder = oFso.GetFolder(sFolderPath)

Dim sFilePath: sFilePath = oFso.GetAbsolutePathName("!" & oFolder.Name & ".txt")

'Declaration of Un-Initialized Work Variables

Dim oSubFolder, aFiles

'Declaration of Work Variables with Initial Values

Dim iFolderFileCount: iFolderFileCount = 0

Dim iFolderByteCount: iFolderByteCount = 0

Dim iFileCount: iFileCount = 0

Dim iByteCount: iByteCount = 0

Dim sFileList: sFileList = ""

'Check if an old Listings File Already Exists and Delete it

if oFso.FileExists(sFilePath) then

   oFso.DeleteFile sFilePath, true

end if

Dim oFile: Set oFile = oFso.OpenTextFile(sFilePath, ForAppending, true, -2)

oFile.WriteLine ""

oFile.WriteLine "Content of: " & oFolder.Name

oFile.WriteLine replace(space(SeparatorLength)," ","=")

oFile.WriteLine ""

'Process Sub Folders of Current Work Folder

For each oSubFolder in oFolder.SubFolders

   if oSubFolder.Name = "0-9" or _

      ( len(oSubFolder.Name) = 1 and asc(left(oSubFolder.Name,1))>= 65 and _

      asc(left(oSubFolder.Name,1))<= 90 ) then

   'Only process Sub Folders "0-9", "A","B" ..."Z" and ignore the rest

    Set oFolderFiles = oSubFolder.Files

      'Get Files in Sub Folder and Build List

    For each oFolderFile in oFolderFiles

         'oFso.GetFileName

       sFileList = sFileList & oFolderFile.Name & vbcrlf

         iFolderFileCount  = iFolderFileCount + 1

         iFolderByteCount = iFolderByteCount + oFolderFile.Size

       iFileCount = iFileCount + 1

         iByteCount = iByteCount + oFolderFile.Size

    Next

     oFile.WriteLine replace(space(SeparatorLength)," ","-")

     oFile.WriteLine "Folder: " &  oSubFolder.Name

     oFile.WriteLine replace(space(SeparatorLength)," ","-")

     oFile.WriteLine "Total: " &  formatnumber(iFolderFileCount,0) & " files"

     oFile.WriteLine "Total Size: " &  formatnumber(iFolderByteCount,0) & " bytes" & _

                     " (" & formatnumber(iFolderByteCount/1024,0) & " KB / " & _

                     formatnumber(iFolderByteCount/1024000,0) & " MB)" 

     oFile.WriteLine replace(space(SeparatorLength)," ","-")

     oFile.Write sFileList

     oFile.WriteLine ""

     sFileList = ""

     iFolderFileCount = 0

     iFolderByteCount = 0

   end if

Next

oFile.WriteLine replace(space(SeparatorLength)," ","=")

oFile.WriteLine oFolder.Name

oFile.WriteLine "Total: " &  formatnumber(iFileCount,0) & " files"

oFile.WriteLine "Total Size: " &  formatnumber(iByteCount,0) & " bytes" & _

                " (" & formatnumber(iByteCount/1024,0) & " KB / " & _

                formatnumber(iByteCount/1024000,0) & " MB)" 

oFile.WriteLine ""

oFile.Close

Set oFile = Nothing

Set oFolder = Nothing

Set oFso = Nothing

I hope that you might also find use for those little but helpful scripts. If not, I hope that you enjoyed my little personal trick back in time about my collecting habits. If you are a collector like me, then you can probably relate to those things without any problems. If you are not, then you might got a nice laugh out of it instead. hehe.

Cheers!

Carsten aka Roy/SAC