| At line 1 changed one line |
| This plugin allows you to post process files using a powerful set of tasks. You can move files, rename them, copy them, execute external programs, write text files, use date variables, multithread operations, copy to FTP/FTPS/SFTP servers, etc. |
| __IMPORTANT: due to the security updates since CrushFTP version 10.5.2+ any JDBC driver jar file needs to be placed into the CrushFTP10/plugins/lib/ directory, or it won't load.__ |
| \\ |
| \\ |
| This plugin allows you to post-process files using a powerful set of tasks. You can move files, rename them, copy them, execute external programs, write text files, use date variables, multithread operations, copy to FTP/FTPS/SFTP servers, etc. |
| At line 3 changed one line |
| Additional info coming soon... |
| [CrushTask Functions] helps explain some of the advanced string manipulation functions you can do in CrushTask, typically in a UserVariable task. |
|
| [Example Task Flow -- Folder monitor triggered task.|CrushTaskExample]\\ |
| [Example Task Flow -- Find local files and copy them to remote servers.|CrushTaskExample2]\\ |
| [Example Task Flow -- Expire users password and email password reset link.|CrushTaskExample3]\\ |
| [Example Task Flow -- Import users from *.csv file.|CrushTaskExample4]\\ |
| [Example Task Flow -- Allow username lookup on internal share.|CrushTaskExample5]\\ |
| [Example Task Flow -- Delete subfolders but keep root folders.|CrushTaskExample6]\\ |
| [Example Task Flow -- Self register and enable accounts.|CrushTaskExample7]\\ |
| [Example Task Flow -- Notify users of expired password.|CrushTaskExample8]\\ |
| [Example Task Flow -- Auto register and set password on first logon.|CrushTaskExample9]\\ |
| [Example Task Flow -- Auto Update CrushFTP.|CrushTaskExample10]\\ |
| [Example Task Flow -- Upload with approval and move files.|CrushTaskExample11]\\ |
| [Example Task Flow -- Looping through files and delete copied files.|CrushTaskExample12]\\ |
| [Example Task Flow -- Find new files and send a notification or process them.|CrushTaskExample13]\\ |
| [Example Task Flow -- Copy or move folders to another location preserving the inner folder structure (Recursive Copy/Move)|CrushTaskExample14]\\ |
| [Example Task Flow -- Find and Copy items older than and newer than X days|CrushTaskExample15]\\ |
| [Example Task Flow -- One-click upload and notify a contact about it|CrushTaskExample16]\\ |
| [Example Task Flow -- Setting up a Controller job for multiple Data Centers and redundancy|CrushTaskExample17]\\ |
| [Example Task Flow -- Renew Azure SAS token via Azure User impersonation| CrushTaskExample18] |
|
| The source filter is on every task. It will filter out items that don't match the source from being included in the current task item. So you may want to apply certain actions to specific file types on a per-task basis. |
|
| MultiThreaded allows for tasks to run in parallel. Be sure to add a Wait task to have things stop and wait for all the separate threads that a re running before allowing the entire job to complete. This is useful to transfer a file to multiple external locations at the same time instead of sequentially. |
|
| The error jump field allows you to jump to a particular task item in the list if the current task fails for any reason.\\ |
| __Default Failure__ : If a task has this name, then that task will capture all failures without needing to draw lines to it.\\ |
| ---- |
| !Find |
| The find task will scan a particular folder or remote FTP, or SFTP server getting directory listings recursively up to the depth allowed. These found items can then be used by future task items. This task is often the starting item in a list of tasks. |
|
| You can use Regular Expressions in the Find Filter. Regular expressions need to be prefixed with "REGEX:" literal. Also, keep in mind we do a match against the full source URL. Finally, we only support Java-style regular expressions. |
|
| You can enable verification that the files are not changing to prevent getting files that are still being written to. If you want to preserve the folder path to the files for a future copy, enable the "Use absolute paths" option. |
|
| The last modified selection will only find files that are the specified age, either newer or older. |
|
| If no items have been found, it can continue scanning for up to the maximum minutes defined, with a pause of the defined number of seconds between scans. If at the end of all of this, no files are found, you can cause the task to fail with the "Fail if no files are found" checkbox. |
|
| If the FindCache option is used, the items found will further be filtered out so prior known items aren't found again. |
|
| [attachments|find_task.png] |
| ---- |
| !FindCache |
| This cache task allows you to load in a prior listing of items to be compared with a new listing. This allows you to filter out prior known items that have already been processed without ever altering them on the remote server. |
|
| You first do a "read" mode on the FindCache, specifying the name. Then in the copy task, you specify that same name. If the Find task item finds an item already in the list, then its ignored. |
|
| Next you would do another FindCache and do a "write" mode so it saves the list back to disk with the updated listing. Any items the FindCache knew about that are no longer on the server are removed from the list that is saved as well. |
|
| [attachments|findcache_task.png] |
| ---- |
| !Copy |
| The copy task will take a collection of files that were either "Found" using the Find task, or passed into the plugin through an event defined in the user manager. The files in the list can be filtered down using the "Source filter" field to limit what you are copying. An example might be *.txt. The destination can reference the {path} to the original file. |
|
| So you may do: /Archive/{path} or /Archive/{parent_path}/{name} |
|
| The wait amount is how long to wait to verify the file is no longer being written to by some outside process. The modified date and file size are both compared. It will wait the maximum amount of time before giving up and aborting the job. |
|
| If you want to work with the copied files, you can have them added to the list of files that are being processed for future tasks. Be careful with this because if you do this, and then do a delete, both the original and copy would then be deleted. |
|
| [attachements|copy_task.png] |
| ---- |
| !Move |
| The move task is similar to the copy task but it will just move the file to the destination location. If the move is across physical locations, it will copy, and then delete the source item to accomplish the move. |
|
| [attachments|move_task.png] |
| ---- |
| !Delete |
| The delete task will delete items that are in the list of files being worked with. So this includes items from the "Find" task, or items passed to the plugin through an event. It will keep attempting a delete for x seconds in the event the file is in use, or locked, etc. |
|
| [attachments|delete_task.png] |
| ---- |
| !Email |
| The email task can send customized emails out. These emails could contain the custom form fields from an upload of a web browser, or information about the files being uploaded. <LINE>{name}</LINE> can be used to repeat for each file item being listed. Files can also be attached to the email message as well. (small files) |
|
| The body can also generate temporary share links. Examples:\\ |
| http://domain.com/{share_move_30}\\ |
| http://domain.com/{share_move_15}\\ |
| http://domain.com/{share_reference_45}\\ |
| http://domain.com/{share_copy_7}\\ |
| http://domain.com/{share_copy_22}\\ |
|
| Where the middle part is the share type, and the ending part is the number of days. |
|
| [attachments|email_task.png] |
| ---- |
| !Execute |
| This task allows for running an external process. This can allow outside integration processes to be performed. The output of these programs are fed into the log for this task to allow for monitoring of its results. |
|
| [attachments|execute_task.png] |
|
| !input/parameters: |
| *__Command:__ the name of (or full path to) the external binary. In case running scripts, here comes the script interpreter binary name, cmd.exe for a Windows batch script\\ |
| or sh , /bin/bash , etc. for a linux shell script, etc.\\ |
| *__Argument:__ the list of arguments passed to Command, the list items separated by __Separator__ character ";" (semicolon) by default. In case running scripts, here comes\\ |
| the script file name followed by any arguments to be passed to |
| *__Working directory:__ the parent path of the binary or script file, we do a change directory into this folder before invoking Command. In case running scripts, here comes\\ |
| the full parent path of the script file |
| *__Environment variables:__ leave it empty, rarely needed\\ |
| *__Separator:__ by default ; (__semicolon__), needed due to the specific way white spaces are handled. Need to use this instead of whitespace characters in the __Arguments__ list,\\ |
| those will be replaced at run time with white spaces. |
| !output: |
| the task will feed it's output to the next task item, if any, can be referenced by the {execute_log} server variable. Alternatively, you can always reference {last_execute_log} which is the last one for that entire job, not specific to one particular one. This is useful when just calling execute and not passing in a list of items to that task. |
| ---- |
| !Exclude |
| The list of items being processed is passed from task to task, growing when you use the Find task to add more. The Exclude task allows you to remove these items from further processing. once excluded, they are no longer referenced and future task steps will not process them in any way. The source filter is used to find the items to exclude. So excluding "*" would remove all items from the list. |
|
| [attachments|exclude_task.png] |
| ---- |
| !MakeDirectory |
| This allows you to create a new folder at the specified location. During normal copy operations, directories are created as needed. |
|
| [attachments|makedirectory_task.png] |
| ---- |
| !Preview |
| This task will instantly pass in any found files to the [preview] worker inside of CrushFTP requesting for the files to have their thumbnails generates. This is only useful if you are showing previews of image-type files on the [WebInterface]. |
|
| [attachments|preview_task.png] |
| ---- |
| !Rename |
| This task will rename a file in its current location to have a new name. |
|
| [attachments|rename_task.png] |
| ---- |
| !Jump |
| This task will jump to a specified job, and task item. If the job is left empty, then the task in the current job will be located and used. If a job is jumped to, the task that matches the name will be the starting point in that job. Jumping to a job is only possible for enterprise licenses. Once the jumped to job is complete, the task will continue on to the next step of the current job.\\ |
| Jump also has the special ability to "group" matching items together and call the "true" task with each grouping of items. This might be useful for example to group files together based on extension, or based on their modified date's Month, etc. The syntax for grouping is to use the left side condition of "GROUPBY" then matches pattern, then use some expression on the right side like {MMM} for example. |
|
| [attachments|jump_task.png] |
| ---- |
| !UserVariable |
| This allows you to make a variable that you will then reference in other future steps. So you might make a "archive" variable and then reference it in other steps as {archive} so that if that changed, you would only need to change it in one location. |
|
| [attachments|uservariable_task.png] |
| ---- |
| !UnZip |
| This will unzip a file that is in the list. It's suggested you filter down on *.zip to only get valid items in case of a mixed list of items. The external unzip method will call the OS's unzip utility to unzip. This will only work on OS X, or Linux / Unix-based systems. Otherwise, the internal method will work too. |
|
| [attachments|unzip_task.png] |
| ---- |
| !Zip |
| This will take all items in the list and zip them into a single file. It's suggested to then do an Exclude task to them remove all items, and then go Find the single zip you just made if you want to do further processing. |
|
| [attachments|zip_task.png] |
| ---- |
| !Wait |
| This task will pause for the requested number of seconds before going to the next step. You can also enable the wait for previous threads, and it will wait indefinitely until the prior MultiThreaded tasks have all completed. |
|
| [attachments|wait_task.png] |
| ---- |
| !WriteFile |
| This task will write to a log file, or xml file, or whatever you like, information about the files being processed. One example is to create an RSS feed of incoming files. The RSS configuration is pre-filled in the fields by default. You can have the task find a specific piece of text starting from the beginning of the file, and then insert the new data there, or find a specific piece of text going from the end of the file, and then append data there. If the file doesn't exist yet, the New File section will be written to the file before it performs the searches and updates to the file. |
|
| [attachments|writefile_task.png] |
| ---- |
| !AS2 |
| This task allows for taking the items in the list and sending them on to a AS2 server. You can configure all the typical AS2 settings for encryption and signing.\\ |
| The recipient URL can also have custom headers defined. Example:\\ |
| ''Example: https://other_domain.com/as2#Contet-Type=text/plain&Another-Header-Name=something else&etc'' |
|
| [attachments|as2_task.png] |
| ---- |
| !HTTP |
| The HTTP task lets you post events to another HTTP server with information about files that were transferred. You can control various aspects about how the connection is made and put in your variables for the data you want to be posted. You have some additional variables like these to reference: {http_response_code}, {http_response_message}, {http_response_log}. |
|
| [attachments|http_task.png] |
| ---- |
| !PGP |
| Files can be encrypted, or decrypted using PGP public/private keys. |
|
| [attachments|pgp_task.png] |
|
| __New Method: Info__ -> Usage : If the input is a pgp public key file and the Method is "Info" the PgpTask will add the pgp key settings info, and you can reference them as variables like: |
| {{{ |
| {pgp_user_id},{pgp_creation_time},{pgp_is_expired}, {pgp_expiration_date},{pgp_expiration_time}, {pgp_algorithm},{pgp_algorithm_type},{pgp_prefferred_cipherse},{pgp_prefferred_hash},{pgp_ascii_version_header},{pgp_valid_days},{pgp_key_size},{pgp_key_id_long_hex},{pgp_is_valid_forever} |
| }}}\\ |
| \\ |
| __PBE (Password-based encryption)__\\ |
| \\ |
| Set the key location to "password:".\\ |
| Provide the password in the "Key password(if any)" input field.\\ |
| ---- |
| !POP3 / IMAP |
| This task allows you to scan a mail server finding matching emails based on subjects or senders, and pull out attachments from them. You can optionally delete the messages after your done too. These attachments can then be processed with CrushTask in additional methods. This task type is only allowed for Enterprise licenses. |
|
| [attachments|popimap_task.png] |
|
| Here is a list of other possible variables: |
| subject, body, from, mail_index, mail_type, url, the_file_name, the_file_path, modified, from_prefix. |
|
| ---- |
| !Users List |
| This task can iterate through your users, and make calls to another task item for each user in the list. A good use for this might be the popimap task if you wanted to monitor the user's email server and give them files that had arrived for example. |
|
| [attachments|userslist_task.png] |
| ---- |
| !SQL |
| This task can be used to run SQL statements or even full scripts against an SQL database server. The DB driver class name and the URL syntax is usually specific to a certain DB family, need to find out from the documentation of the JDBC driver. |
|
| [{Image src='sql_task.jpg' width='..' height='..' align='left' style='..' class='..' }] |
|
|
| !parameters: |
| *DB Driver: the JDBC driver class name.\\ |
| *DB Driver File: the JDBC driver jar file path on server.\\ |
| *DB URL: the SQL server URL we connect to.\\ |
| *DB User, DB Password: the login credentials of the DB query account.\\ |
| *Query: input field for the SQL statement or script.\\ |
| The __Rollback if a transaction fails__ flag if set, the task signals the SQL server before issuing the transaction to automatically roll back in case of a runtime error was raised. |
| \\ |
| !input: |
| Generic list items passing from the previous task. Input fields also accept server variables, where it makes sense. |
| !output: |
| The SQL task will feed it's output to the next task item, if any. The DB table column names can be used as server variables, enclosed within curly brackets, to reference table cell values of a row. The next task will usually loop through the rows list. |
| ---- |
| !Java |
| Here is a sample class that needs to be in the class path. You then reference it directly in the job. You need to include the library for CrushTask.jar for this. Then in your java class, reference "task.MyClass" |
|
| {{{ |
| package tasks; |
|
| import java.util.Properties; |
| import java.util.Vector; |
|
| public class MyClass extends Task |
| { |
|
| public MyClass(Properties prefs, Vector items, Properties info, Vector threads) |
| { |
| super(prefs, items, info, threads); |
| } |
|
| public Task go() |
| { |
| run(); |
| return null; |
| } |
|
| public void run() |
| { |
| try |
| { |
| newItems.addAll(items);//keep all the items for the next task. |
| for (int x = 0; x < items.size(); x++) |
| { |
| Properties p = (Properties) items.elementAt(x); |
| System.out.println(p.getProperty("url")); |
| } |
| Vector v = (Vector) prefs.get("prop_item"); |
| Properties prop_item = new Properties(); |
| for (int x = 0; x < v.size(); x++) |
| { |
| Properties p = (Properties) v.elementAt(x); |
| prop_item.put(p.getProperty("key"), p.getProperty("val", "")); |
| } |
| Enumeration keys = prop_item.keys(); |
| while (keys.hasMoreElements()) |
| { |
| String key = keys.nextElement().toString(); |
| String val = prop_item.getProperty(key, "").trim(); |
| } |
| msg("MyClass"); |
| } |
| catch (Exception e) |
| { |
| e.printStackTrace(); |
| msg(e); |
| error = e; |
| } |
| } |
| } |
| }}} |
| ---- |
| !Custom |
| This is a placeholder for custom tasks developers make in the future. |
