Getting rclone issues while running scylla-manager commands

ScyllaDB
1

Hello,
We are getting rclone exceptions while running scylla manager commands in GCP VM’s.
Exceptions :

root@scyl-deng-xp-ab1v1-prd-ase1:/home/chaitanyatondlekar# scylla-manager-agent download-files -L gcs:gcs-deng-ab-scyllabackup-prd --list-snapshots
{"L":"ERROR","T":"2024-01-29T11:21:08.137Z","N":"rclone","M":"parse instance region: invalid character '<' looking for beginning of value","S":"github.com/scylladb/go-log.Logger.log\n\tgithub.com/scylladb/go-log@v0.0.7/logger.go:101\ngithub.com/scylladb/go-log.Logger.Error\n\tgithub.com/scylladb/go-log@v0.0.7/logger.go:84\ngithub.com/scylladb/scylla-manager/v3/pkg/rclone.RedirectLogPrint.func1\n\tgithub.com/scylladb/scylla-manager/v3/pkg/rclone/logger.go:19\ngithub.com/rclone/rclone/fs.LogPrintf\n\tgithub.com/rclone/rclone@v1.51.0/fs/log.go:152\ngithub.com/rclone/rclone/fs.Errorf\n\tgithub.com/rclone/rclone@v1.51.0/fs/log.go:167\ngithub.com/scylladb/scylla-manager/v3/pkg/rclone.awsRegionFromMetadataAPI\n\tgithub.com/scylladb/scylla-manager/v3/pkg/rclone/aws.go:42\ngithub.com/scylladb/scylla-manager/v3/pkg/rclone.(*S3Options).AutoFill\n\tgithub.com/scylladb/scylla-manager/v3/pkg/rclone/options.go:133\ngithub.com/scylladb/scylla-manager/v3/pkg/rclone.RegisterS3Provider\n\tgithub.com/scylladb/scylla-manager/v3/pkg/rclone/providers.go:52\nmain.setupCommand\n\tgithub.com/scylladb/scylla-manager/v3/pkg/cmd/agent/setup.go:35\nmain.glob..func5\n\tgithub.com/scylladb/scylla-manager/v3/pkg/cmd/agent/download_files.go:71\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/cobra@v1.1.1/command.go:850\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/cobra@v1.1.1/command.go:958\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/cobra@v1.1.1/command.go:895\nmain.main\n\tgithub.com/scylladb/scylla-manager/v3/pkg/cmd/agent/main.go:12\nruntime.main\n\truntime/proc.go:250"}
2

@Michal_Leszczynski can you please assist?

3

Hello, looks like some config mismatch, see rclone.awsRegionFromMetadataAPI part. Perhaps rcone thinks it’s AWS cluster not GCP?

1 Like
4

Hi,

this is a know Scylla Manager issue (see `scylla-manager-agent check-location` command fails to handle region · Issue #3691 · scylladb/scylla-manager · GitHub) that will be fixed in next releases.
When there is no region specified in agent’s AWS config, Scylla Manager tries to fetch this information and fails (because of running on GCP). This indeed prints an error log, but it shouldn’t have an effect on the actual procedure.
So does SM work correctly despite this error in logs?

5

Exact which scylla manager release it will be fixed. ?
We have tried with 3.0 and 3.1 both and both had same exceptions.
This is impacting actual procedure while doing restoration through ansible.
I am following this : Restore | ScyllaDB Docs

6

I hope it will be a part of 3.2.6 but I don’t know if it will be backported to older 3.0, 3.1.
Generally speaking, the advised way of restoring data (present since SM 3.1) is via SM restore task (Restore | ScyllaDB Docs).

1 Like