PSU Git sync not staging/committing/pushing files

PowerShell Universal
1

The built-in Git sync function in PSU is not staging (and committing/pushing) files. If I go onto the PSU server and open a Git client, I can see all of the modified files in an unstaged state. I can manually stage/commit/push the files, but that defeats the point. I had been using a Bash script to automate the Git sync processes because of issues like this with past versions of PSU, but decided to give it another shot only to see it’s still not working right for some reason. Any ideas?

Product: PowerShell Universal
Version: 4.2.17
2

Went back to using my Bash script for the time being.

1 Like
3

If I can get some help figuring out why our file modifications/additions/deletions are not being seen by PSU for it to then stage/commit/push them, that would likely allow me to use the built-in Git sync again.

The issue seems to be largely (if not entirely) limited to a script we use that downloads config files from some of our clients’ firewalls, and it stores these files directly in PSU’s repository folder. The way the script is designed, it overwrites any already-existing file for the particular unit. If I make a change to any of those files WITHIN PSU (like, using the script editor interface), and save those changes, it sees the file has been modified and the Git sync works, so the Git functionality is not completely broken - it seems to just be a problem of HOW we’re changing the files in the repository.

4

I think I found the issue, but I’m testing to see if I’m right.

1 Like
5

This is the git sync logic:

git pull
if conflicted, update git sync status to conflicted and stop sync
git push
git diff-tree
re-run configuration files that have changes to update resources

We don’t do a stage or commit during the git sync. If you are doing changes outside of the API or admin console, you will have to do that manually.

6

@adam How are you pushing without first staging and committing the files?

Here’s the exact script in question:

param(
    [String]$HuduDeviceID,
    [String]$HuduCompanyID,
    [String]$PaloAltoDeviceIP,
    [String]$HuduCredentialID
)


$Date = Get-Date -format "yyyy-MM-dd-hh-mm-tt"
Import-Module HuduAPI
New-HuduApiKey $Secret:ASIHuduAPIKey
New-HuduBaseURL $HuduBaseDomain
$Device = Get-HuduAssets -Id $HuduDeviceID

$GitFilePath = "C:\ProgramData\UniversalAutomation\Repository\Device Backups" + "\" + $($Device.company_name) + "\" + $($Device.name)
$HuduFilePath = "C:\ProgramData\UniversalAutomation\Temporary Storage" + "\" + $($Device.company_name) + "\" + $($Device.name)
$TemporaryStorageBasePath = "C:\ProgramData\UniversalAutomation\Temporary Storage" + "\" + "*"

$password = Get-HuduPasswords -id $HuduCredentialID
$asset=$password.asset_password
$PaloAltoAPIKey = $asset.password

If(!(Test-Path -PathType container $GitFilePath))
{
      New-Item -ItemType Directory -Path $GitFilePath

}
If(!(Test-Path -PathType container $HuduFilePath))
{
      New-Item -ItemType Directory -Path $HuduFilePath

}
$XMLHudu = $HuduFilePath + "\" + $Date + "-" + $($Device.Name) + ".xml"
$XMLGit = $GitFilePath + "\" + $($Device.Name) + "-Running-Config.xml"

$WebRequest = Invoke-WebRequest -SkipCertificateCheck -Uri "https://$PaloAltoDeviceIP/api/?type=export&category=configuration&key=$PaloAltoAPIKey"

# If the HTTP response is not 200 (OK), send the message
if ($WebRequest.StatusCode -ne 200)
{
	#Send-Webhook
    $webhookheaders = @{Authorization = "$Secret:ASI-FS-WebhookAPIKey"}
    $webhookbody = @{
    message = $Device.Name + ' Device Backup Failure'
    subject = $Device.Name + ' Device Backup Failure'
   resource = $Device.Name
   severity = 'minor'
  clientname = Device.company_name
} | ConvertTo-Json
$response = Invoke-RestMethod $AlertWebhookURL -Method 'POST' -Headers $webhookheaders -Body $webhookbody
$response
Write-Error "Error Connecting to Firewall"
}
else
{
        $WebRequest.Content | Out-File -FilePath $XMLGit
        $WebRequest.Content | Out-File -FilePath $XMLHudu
        New-HuduUpload -FilePath $XMLHudu -RecordId $HuduDeviceID -RecordType 'Asset'
        Remove-Item $TemporaryStorageBasePath -Force -Recurse
        Get-HuduUploads | Where-Object {$_.name -like "*" + $Device.Name + "*" -and $_.ext -eq 'xml'} | Select-Object -skip 2 | Remove-HuduUpload -Confirm:$false
        }

It’s the $WebRequest.Content | Out-File -FilePath $XMLGit line that saves the output to a file inside of the PSU repository.

Are you saying that because we’re not writing the file directly inside of PSU (we’re doing it inside of PSU as a scheduled job that runs the script, which is also inside of PSU), that PSU has no idea that a file has been changed and to include it in the Git sync?

7

Commits and staging happens when the file is edited so that we can correctly set the author of the commit based on the user doing so.

Since the git sync runs in the background, it would have the author set to the machine account if we did the staging there.

You could just do the following at the end of your script to have it pick up everything you changed.

git add . 
git commit -m "Something" 
git stage .
1 Like
8

I see. Thanks for clarifying the way you’ve got PSU designed to do the Git syncing, and the workaround. I’ll give that a shot instead of using the Bash script.

Edit: The addition of the git commands at the end of the script did resolve the problem of PSU not seeing the changes, however I also had to add safecrlf = false under the [core] section of .git/config to prevent the job in PSU from displaying the warning message CRLF will be replaced by LF the next time Git touches it while still allowing it to make that EoL modification.

1 Like
9

The new script, in case anyone is curious or wants to use it, is:

param(
    [String]$HuduDeviceID,
    [String]$HuduCompanyID,
    [String]$PaloAltoDeviceIP,
    [String]$HuduCredentialID
)


$Date = Get-Date -format "yyyy-MM-dd-hh-mm-tt"
Import-Module HuduAPI
New-HuduApiKey $Secret:ASIHuduAPIKey
New-HuduBaseURL $HuduBaseDomain
$Device = Get-HuduAssets -Id $HuduDeviceID

$GitFilePath = "C:\ProgramData\UniversalAutomation\Repository\Device Backups" + "\" + $($Device.company_name) + "\" + $($Device.name)
$HuduFilePath = "C:\ProgramData\UniversalAutomation\Temporary Storage" + "\" + $($Device.company_name) + "\" + $($Device.name)

$password = Get-HuduPasswords -id $HuduCredentialID
$asset=$password.asset_password
$PaloAltoAPIKey = $asset.password

If(!(Test-Path -PathType container $GitFilePath))
{
      New-Item -ItemType Directory -Path $GitFilePath

}
If(!(Test-Path -PathType container $HuduFilePath))
{
      New-Item -ItemType Directory -Path $HuduFilePath

}
$XMLHudu = $HuduFilePath + "\" + $Date + "-" + $($Device.Name) + ".xml"
$XMLGit = $GitFilePath + "\" + $($Device.Name) + "-Running-Config.xml"

$WebRequest = Invoke-WebRequest -SkipCertificateCheck -Uri "https://$PaloAltoDeviceIP/api/?type=export&category=configuration&key=$PaloAltoAPIKey"

# If the HTTP response is not 200 (OK), send the message
if ($WebRequest.StatusCode -ne 200)
{
	#Send-Webhook
    $webhookheaders = @{Authorization = "$Secret:ASI-FS-WebhookAPIKey"}
    $webhookbody = @{
    message = $Device.Name + ' Device Backup Failure'
    subject = $Device.Name + ' Device Backup Failure'
   resource = $Device.Name
   severity = 'minor'
  clientname = Device.company_name
} | ConvertTo-Json
$response = Invoke-RestMethod $AlertWebhookURL -Method 'POST' -Headers $webhookheaders -Body $webhookbody
$response
Write-Error "Error Connecting to Firewall"
}
else
{
        $WebRequest.Content | Out-File -FilePath $XMLGit
        $WebRequest.Content | Out-File -FilePath $XMLHudu
        New-HuduUpload -FilePath $XMLHudu -RecordId $HuduDeviceID -RecordType 'Asset'
        Remove-Item $XMLHudu -Force
        Get-HuduUploads | Where-Object {$_.name -like "*" + $Device.Name + "*" -and $_.ext -eq 'xml'} | Select-Object -skip 2 | Remove-HuduUpload -Confirm:$false
        $GitCommitMessage = $($Device.Name) + " configuration file has been updated"
        git add $XMLGit
        git commit -m $GitCommitMessage
        git stage $XMLGit
}