Polling examples

 

Under some circumstances you may need TextPipe to run whenever a file appears in a folder or directory. This file could be generated by a third party program on the same computer, or by a third party program on a different computer (possibly running Unix). We need to repeatedly check to see if the files we need have appeared. This process is called polling. How often we poll determines how quickly the results will be available. For a near-real-time application we might poll every minute or every 30 seconds. For a batch job we might poll once an hour or every 6 hours.

Or - see our new File and Folder Watcher tool

An Example

Let's say that the folder we are monitoring is E:\. The file we want to process will appear in this folder. E:\ can actually point to a Unix file system folder by using software like Samba or TotalNet Advanced Server to make Unix file systems available to Windows.

We don't want to process the file until it is fully written. If the file is large it is quite possible we will see the file appear before its contents have been fully written. There are several options to avoid this:

1. Move Completed File to Monitored Folder

Have the third party application first write its file to a hidden folder, then move the completed file to the monitored folder, E:\. A move operation always moves the entire file in one block, therefore there is no chance we will see a partial file. This solution is good if there is only one file to process. Here is the batch file we would use:

@echo off
rem Wait for completed file to appear
if exist e:\data.txt goto process
goto skip
:process

"C:\Program Files\TextPipe\TextPipe.exe" "/f=C:\Program Files\TextPipe\MyFilter.fll" e:\data.txt /g /q

:skip

2. Using a Flag File or Semaphore File

Have the third party application write a file of known name (eg finished.txt) to indicate that it has finished. We can write a batch file that waits to see this file appear before it starts processing. This solution is good if there are multiple files to process. Here is the batch file we would use:

@echo off
rem Wait for 'finished.txt' file to appear before processing
if exist e:\finished.txt goto process
goto skip
:process

del e:\finished.txt
"C:\Program Files\TextPipe\TextPipe.exe" "/f=C:\Program Files\TextPipe\MyFilter.fll" e:\*.* /g /q

:skip

3. Multiple folders with different filters

If you have multiple folders with different file types in each (each folder requiring a different TextPipe filter), try the following solution (which is just an extension of the batch file above):

@echo off

rem Folder 1
if exist e:\folder1\finished.txt goto process1
goto skip1
:process1

del e:\folder1\finished.txt
"C:\Program Files\TextPipe\TextPipe.exe" "/f=C:\Program Files\TextPipe\filter1.fll" e:\folder1\*.* /g /q

:skip1


rem Folder 2
if exist e:\folder2\finished.txt goto process1
goto skip2
:process2

del e:\folder2\finished.txt
"C:\Program Files\TextPipe\TextPipe.exe" "/f=C:\Program Files\TextPipe\filter2.fll" e:\folder2\*.* /g /q

:skip2

...etc