FTPGETDIFF         Get specific changes within FTP site file system

<< Click to Display Table of Contents >>

Navigation:  Robo-FTP User's Guide > Script Programming > Script Commands > All Script Commands >

FTPGETDIFF         Get specific changes within FTP site file system

Syntax:

FTPGETDIFF

[ dbfile ]

Arguments:

[ dbfile ]

Optional Variable or string specifying a snapshot database file. If omitted, the default value is “snapshot_site.sql”. When [ dbfile ] is specified without a full path the default path is used.

Options:

/commitlast

Mark current difference "done" on the next call to FTPGETDIFF.

 

/incldirs

Allows returning new or deleted directories.

 

/smallerthan=xx

Only retrieve changes for which the file is smaller than the provided file size (in bytes). Also accepts special value "unset" (see section "Special value unset", below).

 

/largerthan=xx

Only retrieve changes for which the file is larger than the provided file size (in bytes). Also accepts special value "unset" (see section "Special value unset", below).

 

/olderthan=xx

Only retrieve changes for which the file's last modified time is older than the provided date-time (using the same format as the built-in %datetime variable). Also accepts special value "unset" (see section "Special value unset", below).

 

/newerthan=xx

Only retrieve changes for which the file's last modified time is newer than the provided date-time (using the same format as the built-in %datetime variable). Also accepts special value "unset" (see section "Special value unset", below).

 

/difftype=xx

Specify the type of changes for which to look.  You can either use this option multiple times or separate the values with a pipe character. See the section regarding %ftpdifffileid below for a list of possible options.

 

/tag=xx

(For advanced users only.) Retrieve only changes corresponding to rows in the diff table for which tag column matches the provided tag or is NULL. Any such rows will subsequently be marked with the given tag if not already marked as such.

 

/clearnonmatching

(For advanced use) Removes any pre-existing entries from the diff table in the snapshot database that do not match the specified path.

 

 

The FTPDIFF and FTPGETDIFF commands work together to identify and process changes to files on a remote server.

 

The FTPGETDIFF command sets the values of seven internal variables with details describing a change to a single file on a remote site. These details are pulled from a snapshot database where they were inserted by a previous call to the FTPDIFF command. Subsequent calls to FTPGETDIFF fetch the details of subsequent changes from the database. The FTPGETDIFF command is typically used in a loop so that each detected change is processed during one iteration of the loop. Depending on your needs, the processing may involve one or more of the internal variables set by this command. It is possible for FTPGETDIFF to report file changes in alphabetical order rather than folder hierarchical order depending on how the records are stored in the snapshot database. This command returns $ERROR_READ_EOF when there are no more differences in the database. The FTPDIFFREWIND command is available to reset the file pointer to the first change recorded in the database.

 

If the [ dbfile ] argument is omitted, the default file name “snapshot_site.sql” file is used. An error occurs if the snapshot database file does not exist.

 

The FTPGETDIFF command sets the values of the following variables that you can use in your script logic:

 

%ftpdifffileid

Type of change detected

%ftpdifffiletext

Text description of type of change detected

%ftpdifffilename

Name of changed file

%ftpdifffilepath

Full path of changed file

%ftpdiffnum

Sequential change number

%ftpdiffsize

The size of the file, in bytes

%ftpdiffdatetime

The date and time of the file

 

The possible values of %ftpdifffileid and %ftpdifffiletext are:

 

$DIFF_FILE_NOT_FOUND

5001

** File not found

$DIFF_FILE_IS_NEW

5002

** File is new

$DIFF_FILE_SIZE

5003

** File size changed

$DIFF_FILE_DATETTIME

5004

** File date/time stamp changed

With /incldirs:

 

 

$DIFF_DIRECTORY_NOT_FOUND

5005

** File not found

$DIFF_DIRECTORY_IS_NEW

5006

** File is new

 

To monitor only for specific type(s) of changes, use the /difftype option. To monitor for multiple types of changes, you can either use the /difftype option multiple times (for example, /difftype=$DIFF_FILE_SIZE /difftype=$DIFF_FILE_DATETIME) or separate the values with a pipe character (for example, /difftype=$DIFF_FILE_SIZE|$DIFF_FILE_DATETIME).

 

Here is a simple example that shows how FTPGETDIFF might be used inside a loop to write a text file that lists all the changes found by FTPDIFF:

 

FTPDIFF "*"

:loop

FTPGETDIFF

IFERROR= $ERROR_READ_EOF GOTO done

SET change = "File: " + %ftpdifffilepath + " " + %ftpdifffiletext

WRITEFILE "differences.txt" change /append

GOTO loop

:done

 

The following example loops through all changes and downloads only the new files from a remote site:

 

FTPDIFF "*"

:loop

FTPGETDIFF

IFERROR= $ERROR_READ_EOF GOTO done

IFNUM!= %ftpdifffileid $DIFF_FILE_IS_NEW GOTO loop

RCVFILE %ftpdifffilename

GOTO loop

:done

 

Sometimes it is helpful to start with a fresh database instead of persisting it across runs of a script. This can be achieved by simply preemptively deleting the snapshot database file before calling FTPDIFF. Using this technique allows you to process all files in a directory in a very efficient manner. The following example illustrates how this can be done:

 

SET dbpath = "C:\path\to\database.sql"

DELETE dbpath      ;; preemptively delete dbpath in case it already exists so

                  ;;  so that the database effectively does not persist across runs

FTPDIFF "*" dbpath

:loop

FTPGETDIFF dbpath

IFERROR= $ERROR_READ_EOF GOTO done    

;; process files here:

SET msg = "File " + %ftpdifffilepath + " has size " + %ftpdiffsize + " and was last modified " + %ftpdiffdatetime

CONSOLEMSG msg

GOTO loop

:done

 

 

If you need to restart the loop, use the FTPDIFFREWIND command to reset the file pointer to the first difference recorded in the snapshot database.

 

Normally, all current differences are marked "done" when FTPDIFF is called again. The /commitlast option increases fault tolerance by causing FTPGETDIFF to mark the previously retrieved difference done. This way, a difference should not be processed twice if loop processing is interrupted after FTPGETDIFF is called but before the full loop iteration is complete.

 

Note: Snapshot database files are small SQLite database files created by Robo-FTP to support the DIFF and FTPDIFF family of script commands.

 

Special value unset

 

The /smallerthan=xx, /largerthan=xx, /olderthan=xx, and /newerthan=xx options also accept a special value "unset". When provided, that corresponding option will be effectively disabled. This allows each option to be turned on or off selectively depending on the values of passed in variables. Consider the following usage:

 

 SET upperlimit = "unset" ;; ensure that /smallerthan option is turned off

 SET lowerlimit = 1024

 FTPGETDIFF /smallerthan=upperlimit /largerthan=lowerlimit

 

Important

The success of this command depends on Robo-FTP's ability to automatically read and understand the directory listings returned by the remote site. Most HTTP/HTTPS sites do not return listings in a supported format and many return no raw directory listings in response to HTTP GET requests based on slash-terminated URLs. Some servers respond to HEAD requests and are partially compatible. Some supported HTTP listing types like APACHE_DEFAULT include file sizes rounded to the nearest megabyte or kilobyte and are therefore not fully compatible with this command. Contact technical support if you have an urgent need related to a raw directory listing format that is currently unsupported.

 

 

Related command(s): FTPDIFF, FTPDIFFREWIND, GETDIFF

See also: Processing Dynamic Folders, Using the built-in database engine, Default Path