ScyllaDB dry run test ERROR - giving up after 2 attempts: agent [HTTP 500] init location: Failed to acquire MSI token: MSI is not enabled on this VM

ScyllaDB
1

I wanted to know if you can help I am facing issue testing the dry run backup to azure blob storage
the VMs are on prem so I used storage account name and key to update the yaml config file

I have 3 nodes

I get this error when I run the
sctool backup -c mycluster -L ‘mybackupname’ --dry-run

Error: create backup target: location is not accessible
** MYIP: giving up after 2 attempts: agent [HTTP 500] init location: Failed to acquire MSI token: MSI is not enabled on this VM: Get “http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fstorage.azure.com”: dial tcp 169.254.169.254:80: connect: no route to host - make sure the location is correct and credentials are set, to debug SSH to MYIP and run “scylla-manager-agent check-location -L MYbackupstorage --debug”**

when i run this scylla-manager-agent check-location -L MYbackupstorage --debug on all of the nodes it returned fine also when I run the scylla-manager-agent check-location -L azure: on all nodes it returns fine too without any errors

the debug on each node returns good ending with the deletion of the test as below on all nodes
{“L”:“DEBUG”,“T”:“2023-10-17T13:13:02.875+0100”,“N”:“rclone”,“M”:“Waiting for deletions to finish”}
{“L”:“DEBUG”,“T”:“2023-10-17T13:13:03.229+0100”,“N”:“rclone”,“M”:“test: Deleted”}

Please how can I resolve the challenge of performing the dry run backup test

I also tried to do an adhoc backup same error too
Error: create backup target: location is not accessible
** MYIP: giving up after 2 attempts: agent [HTTP 500] init location: Failed to acquire MSI token: MSI is not enabled on this VM: Get “http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fstorage.azure.com”: dial tcp 169.254.169.254:80: connect: no route to host - make sure the location is correct and credentials are set, to debug SSH to MYIP and run “scylla-manager-agent check-location -L MYbackupstorage --debug”**

Please help

2

This may or not be related but you typically want to schedule your backup by prepending the provider beforehand, as in:

azure:bucket_name

That said, the error you have shown is a DNS lookup error, which makes sense, given that 169.254.169.254 is the IP of Azure’s internal metadata service which - as you are running on-prem - it won’t be able to resolve.

Let’s try to use the correct backup name, and should the problem persist, raise an issue under the ScyllaDB Manager GitHub repo.

3

Hi @felipemendes
Thank you for your response. I wanted to ask like you have said, is it not possible to run the Scylla setup and the manager on the On-Prem VMs while we push the backups to the azure blob storage?

How do I prepend the provider? will I add that (azure:bucket_name) to a file?

I have used the correct commands and followed all guides here in and here Examples | ScyllaDB Docs

I will look into the DNS an

4

It is possible, hence why I explained in my previous post that the process shouldn’t abort upon a DNS failure, but rather likely fallback to resolving the Azure Blobstorage endpoint via normal means.

How do I prepend the provider? will I add that (azure:bucket_name) to a file?

In your command-line, use:
sctool backup -c mycluster -L azure:mybackupname --dry-run

Please follow-up with a Github issue as initially explained should this not resolve your problem.