View Attach (22) Info

Add new attachment

Only authorized users are allowed to upload new attachments.

List of attachments

Kind Attachment Name Size Version Date Modified Author Change note
png
 azure_blob_user_delegation_set... 133.7 kB 1 04-Oct-2024 04:55 krivacsz
png
 azure_oauth_2_data.png 175.6 kB 2 05-Oct-2024 01:47 krivacsz
png
 check_aouth2_http_response.png 153.5 kB 1 05-Oct-2024 01:56 krivacsz
png
 check_azure_refresh_token.png 162.1 kB 1 05-Oct-2024 01:13 krivacsz
png
 check_if_difference_is_less_th... 150.8 kB 1 04-Oct-2024 08:28 krivacsz
png
 check_persist_variable.png 166.4 kB 1 05-Oct-2024 01:05 krivacsz
png
 check_response_code_user_deleg... 153.3 kB 1 05-Oct-2024 03:33 krivacsz
png
 construct_azure_sas_token.png 207.5 kB 1 05-Oct-2024 03:47 krivacsz
png
 default_failure_task.png 70.6 kB 2 05-Oct-2024 03:59 krivacsz
png
 file_parse_azure_vfs_file.png 99.1 kB 1 04-Oct-2024 08:09 krivacsz
png
 find_azure_vfs_file.png 143.4 kB 1 04-Oct-2024 08:05 krivacsz
png
 get_azure_sas_token_info.png 181.0 kB 1 04-Oct-2024 08:12 krivacsz
xml
 job.XML 151.8 kB 1 05-Oct-2024 04:03 krivacsz
png
 jump_difference_greater_then_z... 152.1 kB 1 04-Oct-2024 08:22 krivacsz
png
 oauth2_http_task.png 145.6 kB 1 05-Oct-2024 01:50 krivacsz
png
 renew_azure_sas_token_user_imp... 407.9 kB 1 04-Oct-2024 04:27 krivacsz
png
 replace_azure_sas_token.png 104.4 kB 1 05-Oct-2024 03:51 krivacsz
png
 save_azure_refresh_token.png 92.6 kB 1 05-Oct-2024 01:26 krivacsz
png
 set_difference_as_zreo.png 73.9 kB 1 04-Oct-2024 08:25 krivacsz
png
 user_delegation_parse_result.p... 119.4 kB 1 05-Oct-2024 03:41 krivacsz
png
 user_delgation_http_call.png 159.1 kB 1 05-Oct-2024 03:31 krivacsz
png
 variables_for_delegation_key_c... 184.8 kB 1 05-Oct-2024 02:02 krivacsz

This page (revision-79) was last changed on 07-Oct-2024 05:07 by krivacsz

This page was created on 04-Oct-2024 04:27 by krivacsz

Only authorized users are allowed to rename pages.

Only authorized users are allowed to delete pages.

Difference between version and

At line 10 changed one line
[attachments|azure_blob_user_delegation_settings.png]\\
[attachments|Azure Integration/user_delegation_settings.png]\\
At line 12 changed one line
__Section 1.__\\
!!Section 1.\\
At line 22 added 12 lines
Parse the VFS XML file to retrieve the Azure SAS token expiration.\\
\\
{{{
vfs_xml = {xml_parse_start}{file_contents}{xml_parse_end}
org_sas_token = {decrypt_start}{v_0_sas_token}{decrypt_end}
se_all = {split_start:&se=:1}{org_sas_token}{split_end}
se_end_index = {indexof_start:&:0}{se_all}{indexof_end}
se = {substring_start:0:{se_end_index}}{se_all}{substring_end}
sas_token_expire = {parse_start:yyyy-MM-dd'T'HH~..~mm~..~ss'Z'}{se}{parse_end}
difference = {math_start:l}{sas_token_expire}-{now}{math_end}
}}}\\
\\
At line 24 changed one line
__Section 2.__\\
!!Section 2.\\
At line 26 changed one line
Check the SAS token expires and store the refresh token. The refresh token needs to be stored each time. We store it as a persistent variable. \\
Verify the expiration of the SAS token and store the refresh token. The refresh token must be saved each time and stored as a persistent variable.\\
At line 40 added 130 lines
[attachments|jump_difference_greater_then_zero.png]\\
\\
Set the difference to zero if it is less than zero.\\
\\
[attachments|set_difference_as_zreo.png]\\
\\
Verify if the difference is less than two days.\\
\\
[attachments|check_if_difference_is_less_then_two_day.png]\\
\\
Determine whether the persistent variable is present. On the first run, it will not exist.\\
\\
[attachments|check_persist_variable.png]\\
\\
Check for the presence of the persistent variable.\\
\\
[attachments|check_azure_refresh_token.png]\\
\\
Persist the Azure's refresh token.\\
\\
{{{
persist_azure_refresh_token = {decrypt_start}{v_0_azure_user_delegation_refresh_token}{decrypt_end}
}}}\\
\\
[attachments|save_azure_refresh_token.png]\\
\\
!!Section 3.\\
\\
This section retrieves the Access Token using the Azure Refresh Token to obtain the user delegation key.\\
\\
Construct the request body of the HTTP call.\\
\\
{{{
client_id = {decrypt_start}{v_0_azure_user_delegation_client_id}{decrypt_end}
tenant_id = {decrypt_start}{v_0_azure_user_delegation_client_tenant}{decrypt_end}
client_secret = {decrypt_start}{v_0_azure_user_delegation_client_secret}{decrypt_end}
renew_refresh_token_post_data = client_id={client_id}&client_secret={client_secret}&refresh_token={persist_azure_refresh_token}&grant_type=refresh_token
blob_storage_account_name = {url_start:user}{decrypt_start}{v_0_url}{decrypt_end}{url_end}
}}}\\
\\
[attachments|azure_oauth_2_data.png]\\
\\
Retrieve the Access Token through the HTTP Task.\\
\\
{{{
URL = https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token
POST Data = {renew_refresh_token_post_data}
Content-Type = application/x-www-form-urlencoded
}}}
\\
[attachments|CrushTaskExample18/oauth2_http_task.png]\\
\\
Verify the HTTP response.\\
\\
[attachments|check_aouth2_http_response.png]\\
\\
Parse the HTTP response and build the request body for the HTTP call to get the delegation key. __!!! We must store the new refresh token to maintain offline access.\\
\\
{{{
response = {json_parse_start}{http_response_log}{json_parse_end}
persist_azure_refresh_token = {refresh_token}
after_6_days_in_millis = {math_start:l}{now}+518400000{math_end}
sas_start = {rparse_start:yyyy-MM-dd'T'HH~..~mm~..~ss'Z'}{yesterday}{rparse_end}
sas_end = {rparse_start:yyyy-MM-dd'T'HH~..~mm~..~ss'Z'}{after_6_days_in_millis}{rparse_end}
delegation_key_post_body = <?xml version="1.0" encoding="utf-8"?>{n}<KeyInfo>{n} <Start>{sas_start}</Start>{n} <Expiry>{sas_end}</Expiry>{n}</KeyInfo>
}}}\\
\\
[attachments|variables_for_delegation_key_call.png]\\
\\
!!Section 4.\\
\\
This section retrieves the user delegation key, constructs a new Azure SAS token, and replaces the old value with the new value in the user's VFS file.\\
\\
HTTP call to obtain the user delegation key:\\
\\
{{{
URL = https://{blob_storage_account_name}.blob.core.windows.net/?restype=service&comp=userdelegationkey
Post Data = {delegation_key_post_body}
Headers:
x-ms-version = 2022-11-02
Authorization = Bearer {access_token}
}}}\\
\\
[attachments|user_delgation_http_call.png]\\
\\
[attachments|check_response_code_user_delegation.png]\\
\\
Parse the response of the Azure User Delegation HTTP call. The response XML lacks the 'properties' type, so we include it for proper XML file parsing.\\
\\
{{{
parse_response = {xml_parse_start}{replace_start:<UserDelegationKey:<UserDelegationKey type="properties"}{http_response_log}{replace_end}{xml_parse_end}
permission = rwdl
user_delgation_key = {Value}
}}}\\
\\
[attachments|user_delegation_parse_result.png]\\
\\
Construct the Azure SAS token.\\
\\
{{{
azure_sas_string_to_sign = {permission}{n}{n}{sas_end}{n}/blob/crushftpdatalake2/folderkz{n}{SignedOid}{n}{SignedTid}{n}{SignedStart}{n}{SignedExpiry}{n}b{n}{SignedVersion}{n}{n}{n}{n}{n}https{n}{SignedVersion}{n}c{n}{n}{n}{n}{n}{n}{n}
signature = {sign_hmacsha256_start:{user_delgation_key}}{azure_sas_string_to_sign}{sign_hmacsha256_end}
azure_sas_token = sp={permission}&se={sas_end}&skoid={SignedOid}&sktid={SignedTid}&skt={SignedStart}&ske={SignedExpiry}&sks=b&skv={SignedVersion}&spr=https&sv={SignedVersion}&sr=c&sig={sign_hmacsha256_value}
azure_sas_token_encoded = {encrypt_start}{azure_sas_token}{encrypt_end}
}}}\\
\\
[attachments|construct_azure_sas_token.png]\\
\\
Swap the old Azure Sas token with the newly created one.\\
\\
{{{
Find Text = <sas_token>{v_0_sas_token}</sas_token>
Replace With = <sas_token>{azure_sas_token_encoded}</sas_token>
}}}\\
\\
[attachments|replace_azure_sas_token.png]\\
\\
!!Section 5.\\
\\
This section covers the error handling using the Default Failure Task (see at [Handling Job Failures]).
\\
[attachments|default_failure_task.png]\\
\\
Sample job.XML can be downloaded here: [attachments|job.XML]\\
\\
Version Date Modified Size Author Changes ... Change note
79 07-Oct-2024 05:07 6.753 kB krivacsz to previous
78 07-Oct-2024 05:07 6.825 kB krivacsz to previous | to last
77 07-Oct-2024 05:05 6.753 kB krivacsz to previous | to last
76 05-Oct-2024 04:02 6.746 kB krivacsz to previous | to last
75 05-Oct-2024 03:58 6.678 kB krivacsz to previous | to last
74 05-Oct-2024 03:55 6.632 kB krivacsz to previous | to last
73 05-Oct-2024 03:54 6.568 kB krivacsz to previous | to last
72 05-Oct-2024 03:52 6.502 kB krivacsz to previous | to last
71 05-Oct-2024 03:52 6.496 kB krivacsz to previous | to last
70 05-Oct-2024 03:51 6.453 kB krivacsz to previous | to last
69 05-Oct-2024 03:49 6.317 kB krivacsz to previous | to last
68 05-Oct-2024 03:48 6.253 kB krivacsz to previous | to last
67 05-Oct-2024 03:48 6.25 kB krivacsz to previous | to last
66 05-Oct-2024 03:45 6.199 kB krivacsz to previous | to last
65 05-Oct-2024 03:45 6.198 kB krivacsz to previous | to last
64 05-Oct-2024 03:42 5.498 kB krivacsz to previous | to last
63 05-Oct-2024 03:40 5.444 kB krivacsz to previous | to last
62 05-Oct-2024 03:38 5.453 kB krivacsz to previous | to last
61 05-Oct-2024 03:37 5.352 kB krivacsz to previous | to last
« This page (revision-79) was last changed on 07-Oct-2024 05:07 by krivacsz
G’day (anonymous guest)
CrushFTP11 | What's New

Referenced by
CrushTask

JSPWiki