Monday, May 11, 2009

MS DOS Commands, Wild Cards, Input/Output Redirection and Variables

While I was looking for ways to make my life easier by automating things, I rediscovered the MS DOS batch features of the latest MS DOS release and even more extended version of MS DOS for the early Microsoft Windows 32 bit Operating Systems, like Windows 95, Windows CE and Windows 98 (dubbed MS DOS 7).

dospromptani You can accomplish a lot of things right from the DOS prompt without needing to program fancy Windows Applications or something like that.

Many things where I thought the use of “real” programming languages like Visual Basic or VBScript would be necessary, can be accomplished by using sequences of MS DOS commands in a Batch File (.BAT).

BATCH commands are supported by any Microsoft Windows Operation System right out of the box. You do not have to install anything, not mess around with permissions nor do you require Administrator rights and permission to write, edit or execute batch scripts.

The Basics for Newbies

To learn about the available commands and functions that you can use in BATCH files, open a MS DOS window first. To do that, click on “Start”, then “Run”, type “cmd” and then press the ENTER key on your keyboard.

In the MS DOS window type “help” and then press ENTER. Returned will be a list of commands with a brief description of their purpose right next to each of them. Type “help COMMAND”, where COMMAND stands for any command that was listed by “help”, to get a detailed documentation and description of the individual DOS command. There is actually a BATCH file that was written by Rob Van der Woude, which generates a HTML document from those “help” commands, something like a documentation or reference, if you will. You can download the script source here. Save it as AllHelp.bat and then execute it. When it is done, you should have a new file with the name “allhelp.htm” in the same folder as the batch file itself. You can open the HTML file with any web browser.

That covers the basics. The rest of this post is more advanced and for people who are familiar with the basic MS DOS batch features and syntax.

MS DOS (and Windows) Wild Cards

I will explain the wildcards characters used in MS DOS and also Windows for File and Folder Filtering in commands like the "DIR" command and other file based operations. It causes sometimes for confusion and even confused me that I decided to look it up and also conducted some tests to verify the claims made by various people.

There are two (2) wild card characters that can be used for the file system commands in MS DOS and Windows; the question mark character (?) and the asterix or star character (*).

  • ? Matches any single (1) character. It also matches no (0) characters (none), if it is being used in the leading or trailing position of the filter.
  • * Matches matches any and no characters, regardless of its position in the filter

Here are some examples to illustrate the subtle nuances between each of the two wildcards. Joker_Playing_Card_clipart_image

abc*f   matches abcdf, abcdef and abcf
abc*   
matches abc, abcd and abcde
*def   
matches def, cdef and abcdef

abc?ef  matches abcdef but NOT abcef
abc?   
matches abc and abcd but NOT abcde
?
def   
matches def and cdef but NOT abcdef (and also NOT bcdef)
??def   matches def, cdef and bcdef but NOT abcdef

File System Commands Input and Output Routing

Placeholders used:

  • COMMAND stands for the command line command string, which could be anything from the "DIR *.*" command to "ECHO Hi"
  • SRC stands for Source
  • DEST stands for Destination
  • STDIN (0) stands for Standard Input or channel 0
  • STDOUT (1)  stands for Standard Output or channel 1
  • STDERR (2) stands for Errors or channel 2

STDIN and STDOUT can be various different things, such as:

  • NUL = Nothing (usually used for suppressing any output to anywhere)
  • CON = Console or Screen redirect-detour
  • PRN = Printer (Only works, if a DOS printer is configured on LPT1)
  • AUX = Auxiliary
  • COM1 ... COM9 = Serial Ports 1 to 9
  • LPT1 ... LPT9 = Parallel Ports 1 to 9
  • FileName = a File Name, File Path/Location & File Name
  • Command = another Command

The character “<” (less than) stands for Input, the character “>” (greater than) stands for Output (with overwrite) and 2x “>” (greater than)  = “>>” also stands for Output, but without overwrite (appending).

COMMAND>DEST redirects the Standard Output of COMMAND to DEST (Overwriting Previous Outputs)
COMMAND>>DEST redirects the Standard Output of COMMAND to DEST (Not Overwriting Previous Outputs)
COMMAND<SRC feeds COMMAND via the Standard Input from SRC

If you redirect the output to a File for example, you maybe noticed that sometimes not everything is being redirected into the file and some output still occurs on the screen. The reason for that is that you only redirect the STDOUT channel, but not the STDERR channel, which some commands and applications actually use for their error messages output. If you want to redirect the STDERR output to the same output channel specified as DEST use the following syntax:

COMMAND>DEST 2>&1

COMMAND output ">" redirect to DEST (STDOUT), "space", 2, which stands for STDERR and ">" redirect to “&1”, which stands for STDOUT. (Update 7/18/2009: I updated the post and fixed a typo that I made before. I entered 2>&0 instead of 2>&1, which would crash the Batch, because you would feed the error messages back as input to the command. I apologize for this error.) 

Variables in MS DOS Batch

The basics are clear to most folks, like setting a new environment variable with Set VARNAME = VALUE, returning the current value of VARNAME via SET VARNAME or using the VALUE of VARNAME via %VARNAME% in the batch script code. Also the use of the runtime generated variables %1 … %X for the command line parameters that were passed to the batch script are basic BATCH script knowledge.

As you know, parameter values that include the space (blank) character, must be enclosed in double-quotes (“) to avoid that the BATCH script will interpret the individual segments before and after a space as separate parameters. For example BATCH.BAT C:\Documents and Settings would result in three (3) parameters in the Batch Script, like:

%1 = C:\Documents
%2 = and
%3 = Settings

You have to enter BATCH.bat “C:\Documents and Settings” instead to have the path only appear as one (1) parameter in the batch, which is accessible via %1. The problem is that %1 will also include the double-quotes, which are not really part of the value that you want to pass. In many cases this does not matter or is even good, if you invoke file commands for example, where the parameter should again be enclosed in double-quotes to work properly.

To remove the enclosing (“) (double-quotes) from a parameter, add the character “~” (tilde) after the “%” (percent) sign. For example: %~1

This does not work for regular environment variables that were created via the “SET” command. The following would not work and error out, if you try to execute it:

SET VAR=”VALUE”
ECHO %~VALUE%

Other characters that require that a parameter value will be enclosed in double-quotes are: &()[]{}^=;!'+,`~

Did you know that %0 also exists? %0 stands for the  command  itself. In the examples above, %0 would return BATCH.bat. Note: Calling the script via the command BATCH without the extension “.bat” is also valid, if no other executable with the same base name exists in the same directory or path (e.g. a BATCH.exe or BATCH.com). If the batch file is executed this way, %0 also would not include the extension and simply return the value BATCH.

The SET Command

Typical use would be this: Set Counter = 0  where the value of “Counter” would then be accessible in the batch script (and beyond!) via %Counter%

Using the parameter /A indicates that the variable has a numeric value and should also be treated as such. This is important if you want to do arithmetic operations with a variable such as: set /A Counter+=1

This is especially powerful in combination with the “Delayed Environment Variables Expansion” feature enabled.
See more about this option further down below.

set /P VARNAME=Prompt_String

Displays “Prompt_Sting”, then waits for user input (in DOS, not a Windows Input Box) and stores the entered value in the variable VARNAME once the user finished his input by pressing the ENTER key.

Arithmetic Operators for the SET Command

()                  - grouping
! ~ -               - unary operators
* / %               - arithmetic operators
+ -                 - arithmetic operators
<< >>               - logical shift
&                   - bitwise and
^                   - bitwise exclusive or
|                   - bitwise or
= *= /= %= += -=    - assignment
&= ^= |= <<= >>=
,                   - expression separator

String Substitutions

%PATH:str1=str2%  = Expands PATH and substitutes any occurrences of str1 with str2

String Offsets

%PATH:~10,5% = Expands Path and use 5 characters of the value from position 11 (offset) only

Left & Right String Functions

%PATH:~0,-2% = Expands PATH and uses all but the last 2 characters of the value

Other system environment variables

Other variables than %PATH%, which I used in the previous examples, which are good to know, but also require that EXTENSIONS were enabled via the SETLOCAL ENABLEEXTENSIONS command or via cmd.exe parameter.

  • %CD% = Path to current directory
  • %DATE% = Current Date
  • %TIME% = Current Time
  • %RANDOM% = Expands to a random number between 0 and 32767
  • %ERRORLEVEL% = Returns the current ErrorLevel value
  • %CMDEXTVERSION% = Version of the current command processor extensions
  • %CMDCMDLINE% = original command line that invoked the command processor

Delayed Environment Variables Expansion

Always disabled by default, but may be enabled/disabled via the /V command line switch (/V:ON; /V:OFF) to CMD.EXE or via the command “setlocal enabledelayedexpansion” at run-time. 

Qlemo pointed out correctly via a blog comment that you can also make a change to the system registry, to enabled delayed expansion of environment variables by default, if CMD.EXE is invoked. This can be set on individual user level (HKEY_CURRENT_USER) or machine wide, for all users (HKEY_LOCAL_MACHINE).

Key: HKCU or HKLM\Software\Microsoft\Command Processor
Value Name: DelayedExpansion
Value (REG_DWORD): hex:0x1 to enable or hex:0x0 to disable delayed expansion.

If you don't know how to change the settings in the registry, never mind, because only experienced users should make modifications to the system registry. Changes to the registry can cause the system to crash and worse.

Disabled delayed variables expansion means that variables will be expanded at the time they are parsed, which is bad, if the value of the variable is supposed to change within the batch script itself. Here is a nice example script to illustrate the different behavior.

   1: set VAR=before
   2: if "%VAR%" == "before" (
   3:     set VAR=after
   4:     if "%VAR%" == "after" @echo If you see this, it worked
   5: )

If you run this, the script should return nothing. The second IF statement never can become true, even though you have the “SET” command right before it that should have changed the variable’s value to make the IF statement true. Why? Because %VAR% was already set to “before” and because of that expands to “before” throughout the entire script. Now add at the very top the line “setlocal enabledelayedexpansion” and run the script again. Now you should get the text “If you see this, it worked” back as a result.

Here is another example of the implication of the “setlocal enabledelayedexpansion”  statement.

   1: set LIST=
   2: for %%i in (*) do set LIST=!LIST! %%i
   3: echo %LIST%

This script will not return the list of all files in the current directory. It will only show one file name of the directory. It would work the same way as if you would have made the FOR command statement to:

for %%i in (*) do set LIST=%%i

Add “setlocal enabledelayedexpansion”  at the top of the script again and run it once more.
Another typical use where enabling delayed expansion is crucial, are Counters and other mathematical operations and manipulation of values during the execution of a BATCH script.

The SHIFT Command

The SHIFT Command is probably difficult to get its head around it. The applications are versatile. Shift basically shifts the sequence of parameters to the left. Parameter 2 becomes Parameter 1, Parameter 1 disappears and Parameter 2 would be empty, unless there is another Parameter 3. This is hard to grasp, so here is a short sample batch that illustrates this behavior.

   1: @echo off
   2: Call :testfn 1 2
   3: goto :EOF 
   4: :testfn
   5: echo 1 = %1, 2 = %2
   6: echo shift
   7: shift
   8: echo 1 = %1, 2 = %2

Results:


1 = 1, 2 = 2
shift
1 = 2, 2 =

There are other variables like the ones starting with 2x “%” like %%a or variables that are enclosed in “!” instead of “%”, like !VARNAME! instead of %VARNAME%, but that would require me to make this post much longer than it already is. Check the help for the “FOR”, “SET” and “CMD” commands to learn more about them.

Additional Resources to BATCH Scripting

I hope that you find this post of mine useful. Use the comment section below for comments, feedback, suggestions and expressions of appreciation :). Thank you.


Cheers!


Carsten aka Roy/SAC

4 comments:

Anonymous said...

Carsten,
I always try to at least glance through articles I run across during my Google searches that seem close to what I'm looking for, and I didn't expect much when I started reading yours since I consider myself quite experienced with DOS, but you hit a couple new topics to me and I thought your descriptions were written well.
Thanks,
Ken

Anonymous said...

Carsten,
I just want to add you should tell them that there is a way to set Delayed Expansion permanently via the registry, opposed to setlocal or cmd /v, which are temporary.

Thanks for linking to my Experts-Exchange article.

Qlemo

Carsten a.k.a. Roy/SAC said...

Thanks Qlemo,

I added a paragraph to the article which explains that as well.

p.s. you are welcome (link)

Cheers!
Carsten

Anonymous said...

see those two pages:

http://www.computing.net/answers/dos/how-to-echo-a-/11925.html
(the not so smart way)

http://wiki.answers.com/Q/How_do_you_send_greater-than_bracket_symbols_to_the_screen_in_the_dos_batch_file
(the quite smart way)

Post a Comment

Hi, thanks for taking the time to comment at my blog.

Due to spam issues comments are not immediately posted on the site and require my manual approval first, before they become visible.

I try to approve comments as quickly as possible and usually within 24 hours.

To be notified about follow up comments that are made after yours, use the subscribe option with your email address and you will receive an email alert, if somebody else comments at this post in the future.

Also check out the rest of the website beyond this blog, visit RoySAC.com. Also see my YouTube channels, SACReleases for intros and demos.

Cheers!
Carsten aka Roy/SAC

Note: Only a member of this blog may post a comment.