X Tutup
Skip to content

Add Movistar Cloud backend#9191

Open
lafkpages wants to merge 10 commits intorclone:masterfrom
lafkpages:movistar-cloud-backend
Open

Add Movistar Cloud backend#9191
lafkpages wants to merge 10 commits intorclone:masterfrom
lafkpages:movistar-cloud-backend

Conversation

@lafkpages
Copy link

@lafkpages lafkpages commented Feb 19, 2026

What is the purpose of this change?

This pull request adds a new backend for the Movistar Cloud service.

Was the change discussed in an issue or in the forum before?

Access to Movistar MiCloud vía rclone - Help and Support - rclone forum
Movistar Cloud en Linux | Comunidad Movistar - 5299977

Checklist

  • I have read the contribution guidelines.
  • I have added tests for all changes in this PR if appropriate.
  • I have added documentation for the changes if appropriate.
  • All commit messages are in house style.
  • I'm done, this Pull Request is ready for review :-)

@ncw
Copy link
Member

ncw commented Feb 20, 2026

@lafkpages this looks like it is doing all the right things :-)

Are the integration tests passing? (test_all)

Do they have a free tier for testing do you know?

I gave the CI a kick so we can see what that comes up with.

Thanks

@lafkpages
Copy link
Author

Hi,

The service is free if you are a Movistar client and have a phone line with them, but on its own no, there is no free tier. CI failed due to linting but I think I just fixed that (hopefully).

Running go run ./fstest/test_all -backends movistarcloud fails so I'll get to work on that :)

Server-side copy/move is not yet implemented, which these tests rely on I think.
@lafkpages
Copy link
Author

Are the unit tests meant to take nearly an hour and a half to run? 😅

@ncw
Copy link
Member

ncw commented Feb 26, 2026

The service is free if you are a Movistar client and have a phone line with them, but on its own no, there is no free tier.

If we merge this, I'm going to need an account for the daily integration tests

CI failed due to linting but I think I just fixed that (hopefully).

I just clicked the button to run the CI

Running go run ./fstest/test_all -backends movistarcloud fails so I'll get to work on that :)

Great - let me know if you need help

Are the unit tests meant to take nearly an hour and a half to run? 😅

It depends on the provider. s3 rips through them in about 5 minutes.

For slow backends we might disable some of the tests.

The most important tests are backend/movistar, fs/operations and fs/sync

You can run these manually by cd into the directory and go test --verbose v -remote MyMovistarRemote:

@lafkpages
Copy link
Author

Actually now they take about five hours to run but they all pass 🎉
I'll go ahead and mark this for review.

@lafkpages lafkpages marked this pull request as ready for review February 26, 2026 20:53
@lafkpages
Copy link
Author

If we merge this, I'm going to need an account for the daily integration tests

Five hours a day of integration tests sounds a bit heavy to me... Also, at the moment this is implemented to require an auth cookie in rclone config. However, those don't last very long usually and I'm not sure what the best way to go would be. I believe there's a way to refresh the cookie automatically, but since this then generates a new cookie, it would require modifying the rclone config to persist it, and I'm not sure if that's the best way to go.

@lafkpages
Copy link
Author

@ncw what do you think?

@ncw
Copy link
Member

ncw commented Mar 9, 2026

Actually now they take about five hours to run but they all pass 🎉 I'll go ahead and mark this for review.

OK that is quite slow, but good news :-)

If we merge this, I'm going to need an account for the daily integration tests

Five hours a day of integration tests sounds a bit heavy to me...

Yes, me too! What we do for the slow backends is cut the tests down, so skip the bisync/gitannex tests first and if that is still to slow run only the backend test. You'll see lots of examples in fstest/test_all/config.yaml

Also, at the moment this is implemented to require an auth cookie in rclone config. However, those don't last very long usually and I'm not sure what the best way to go would be. I believe there's a way to refresh the cookie automatically, but since this then generates a new cookie, it would require modifying the rclone config to persist it, and I'm not sure if that's the best way to go.

Storing the cookies in the config file is what the other backends do - users expect rclone to refresh its own tokens. Can you use the oauth framework? It deals with all that for you.

Copy link
Member

@ncw ncw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, I've done a quick review of the code so far - see inline for comments.

This looks like it is nearly there - well done :-)

})
if err != nil {
return err
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reads the entire file into memory before uploading. For large files this will cause excessive memory usage or OOM. Consider using mime/multipart.NewWriter with streaming, or at minimum use the io.Reader directly rather than io.ReadAll. Other backends that do multipart uploads (e.g. backend/fichier) stream the body without buffering everything.

Similarly, the moveByReupload method at line 378 also does io.ReadAll on the downloaded file.

ResponseTime int64 `json:"responsetime"`
Data json.RawMessage `json:"data"`
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These getResponse and postResponse types duplicate api.GetResponse and api.PostResponse defined in api/types.go. Use the types from the api package instead to avoid duplication.

decayConstant = 2 // bigger for slower decay, exponential
rootURL = "https://micloud.movistar.es"
uploadURL = "https://upload.micloud.movistar.es"
userAgent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:147.0) Gecko/20100101 Firefox/147.0"
Copy link
Member

@ncw ncw Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hardcoding a specific Firefox user-agent string is concerning - it impersonates a browser and may violate the service's terms of use. Consider using a generic rclone/v1.74 user-agent (which you will get if you don't add a user-agent), or at least make this configurable. If the API requires a browser-like UA to function, that should be documented.


// Precision return the precision of this Fs
func (f *Fs) Precision() time.Duration {
return time.Second
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Precision returns time.Second but the API uses Unix milliseconds throughout (e.g. ModTimeAsTime divides by 1000). Should this be time.Millisecond?


// Size returns the size of an object in bytes
func (o *Object) Size() int64 {
err := o.readMetaData(context.TODO())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

context.TODO() should not be used here. The Size() method is not expected to do network calls - ideally we'd cache the object size in the Object when the object was created.

)

// Time represents a timestamp from the Movistar Cloud API.
type Time time.Time
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Time type is defined here but appears unused in the backend code — ModTimeAsTime() on Media is used instead for timestamp conversion. Consider removing it if it's not needed, or use it consistently.

Description: "Movistar Cloud",
NewFs: NewFs,
Options: []fs.Option{{
Name: "jsessionid",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The jsessionid option has no OAuth flow or automated token refresh. When the session expires, the backend will fail with opaque errors as you were suggesting.

Consider:

  1. Detecting 401/403 responses in shouldRetry or errorHandler and returning a clear "session expired, please update jsessionid" error message
  2. Implementing proper OAuth if the service supports it ideally using the lib/oauth framework
  3. At minimum, documenting the expected lifetime of the session cookie in the Help text

- MEGA [:page_facing_up:](https://rclone.org/mega/)
- MEGA S4 Object Storage [:page_facing_up:](https://rclone.org/s3/#mega)
- Memory [:page_facing_up:](https://rclone.org/memory/)
- Movistar Cloud [:page_facing_up:](https://rclone.org/movistarcloud/)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Movistar Cloud is listed before the Microsoft entries, which breaks the alphabetical ordering. It should come after the Microsoft block.

delay := 500 * time.Millisecond
const maxDelay = 8 * time.Second
const maxAttempts = 15
for range maxAttempts {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The waitForUpload polling loop doesn't distinguish between a successful completion (status != "V") and exhausting all maxAttempts. After the loop, it always tries to fetch metadata regardless. If all 15 attempts timed out with status still "V", consider returning an error rather than silently proceeding.

rootURL = "https://micloud.movistar.es"
uploadURL = "https://upload.micloud.movistar.es"
userAgent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:147.0) Gecko/20100101 Firefox/147.0"
defaultLimit = 200
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default list_limit is set to 200 here (defaultLimit = 200) but the autogenerated docs in movistarcloud.md show "Default: 1000". These are inconsistent — which value is correct? If the docs were generated from a different version, regenerate them with make backenddocs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

X Tutup