fuel_tools#

Gazebo Fuel Tools: Classes and tools for interacting with Gazebo Fuel

Maintainer: nate AT openrobotics DOT org

GitHub open issues GitHub open pull requests Discourse topics Hex.pm

Build

Status

Test coverage

codecov

Ubuntu Noble

Build Status

Homebrew

Build Status

Windows

Build Status

Gazebo Fuel Tools is composed by a client library and command line tools for interacting with Gazebo Fuel servers.

gazebosim/gz-fuel-tools

Building and installing

See the installation tutorial.

Make sure GZ_CONFIG_PATH is set to the right install location so that gz fuel will work. Default is /usr/local/share/gz.

Examples

For a complete list of commands run gz fuel -h on the command line.

List all models

$ gz fuel list -t model -r | head
https://fuel.gazebosim.org/anonymous/test_model_595389531
https://fuel.gazebosim.org/anonymous/test_model_122023392
https://fuel.gazebosim.org/anonymous/test_model_429486665
https://fuel.gazebosim.org/anonymous/test_model_887243621
https://fuel.gazebosim.org/anonymous/test_model_084900530
https://fuel.gazebosim.org/anonymous/test_model_240061059
https://fuel.gazebosim.org/anonymous/test_model_464734097
https://fuel.gazebosim.org/anonymous/test_model_658598990
https://fuel.gazebosim.org/anonymous/test_model_834617935
https://fuel.gazebosim.org/anonymous/test_model_380348669

Download a model

$ gz fuel download -u https://fuel.gazebosim.org/1.0/OpenRobotics/models/Ambulance -v 4
Downloading model:
  Name: Ambulance
  Owner: openrobotics
  Server:
    URL: https://fuel.gazebosim.org
    Version: 1.0

Download succeeded.

Private access tokens Private models and worlds can be downloaded using access tokens. Access tokens are generated on app.gazebosim.org. After logging in, go to Settings->Access Tokens.

An access token can be used with CLI commands via the --header option:

$ gz fuel download -u https://fuel.gazebosim.org/1.0/openrobotics/models/ambulance --header 'Private-Token: <access_token>'

Or, an access token can be stored in a ~/.gz/fuel/config.yaml file. The token is then automatically used by the command line tool and API calls. Use the configure helper tool create your ~/.gz/fuel/config.yaml file.

$ gz fuel configure

C++ Get List models

  // Create a client (uses https://fuel.gazebosim.org by default)
  gz::fuel_tools::ClientConfig conf;
  gz::fuel_tools::FuelClient client(conf);
  gz::fuel_tools::ModelIter iter = client.Models();
  while (iter)
  {
    std::cout << "Got model: " << iter->Identification().Name() << "\n";
  }

Upload a model

Create an account on https://app.gazebosim.org/ and log in.

Create a private token at https://app.gazebosim.org/settings#access_tokens. Store the generated token someplace safe.

The private token can then be used to upload a model:

gz fuel upload -m ~/path_to_model --header 'Private-token: <TOKEN>'

Dependencies

On ubuntu run

sudo apt install ruby-ffi libzip-dev libcurl-dev libjsoncpp-dev

Roadmap

  • Create the notion of “asset repository” or similar. An asset repository abstracts an entity that can store assets. It can be local or remote. This is the interface for “asset repository”:

    • List(category). E.g.: localRepository.List(“models”) remote1Repository.List(“models”)

    • Details(assetIdentifier). E.g.: Modeldentifier model; model.Owner(“the_owner”); model.Name(“the_name”); localRepository.Details(model) remote1Repository.Details(model)

    • Create(assetIdentifier, path_to_the_asset). E.g.: Modeldentifier model; model.Owner(“the_owner”); model.Name(“the_name”); localRepository.Create(model, path_to_the_asset) remote1Repository.Create(model, path_to_the_asset)

    • Delete(assetIdentifier). E.g.: Modeldentifier model; model.Owner(“the_owner”); model.Name(“the_name”); localRepository.Delete(model) remote1Repository.Delete(model)

    • CopyTo(assetIdentifier, dst_repository). E.g.: Modeldentifier model; model.Owner(“the_owner”); model.Name(“the_name”); localRepository.CopyTo(model, remote1Repository) remote1Repository.CopyTo(model, localRepository)

    • “LocalRepository” and “RemoteRepository” should implement this interface. (Most of the pieces are there, we just need to refactor the code a bit).

  • Think about how to detect when new versions of remote models have been uploaded.

    • Idea of a hash.

  • Add Gazebo Fuel command line utilities for:

    • detail

    • create

    • delete

    • copyTo

  • How to test the client library:

    • Directly against the real backend (staging?)

    • Clone, and compile a local backend?

    • Mocking the backend has the problem of not being in sync with the real backend and missing potential issues.

Gazebo Fuel Tools 10.x

Gazebo Fuel Tools 10.0.0 (2024-09-25)

  1. Baseline: this includes all changes from 9.1.0 and earlier.

  2. Miscellaneous documentation fixes

  3. Update badges to point to gz-fuel-tools10

  4. Ionic changelog

  5. Require cmake 3.22.1

  6. Add missing URI header files

  7. Use Ubuntu 24.04 in Github Actions CI workflow

  8. Remove HIDE_SYMBOLS_BY_DEFAULT: replace by a default configuration in gz-cmake.

  9. Use HIDE_SYMBOLS_BY_DEFAULT

  10. Split ServerConfig and ClientConfig across multiple files

  11. Bumps in Ionic: gz-fuel-tools10

  12. Remove deprecated ignition files

Gazebo Fuel Tools 9.x

Gazebo Fuel Tools 9.1.0 (2024-08-06)

  1. Make CollectionIdentifier::UniqueName consistent

  2. Add Url accessor to Identifiers

  3. Migrate curl_formadd from form API to mime API (deprecated in Ubuntu Noble)

  4. Add package.xml

  5. CLI for creating config.yaml

  6. Clean all ASAN reported memory problems (curl related)

  7. Add Private function to world identifier

  8. Use config.yaml file from default cache location, if it exists

Gazebo Fuel Tools 9.0.3 (2024-04-09)

  1. Use relative install path for gz tool data

Gazebo Fuel Tools 9.0.2 (2024-03-18)

  1. Fix LocalCache so that models/worlds downloaded via fuel.ignitionrobotics.org can be found in the cache

Gazebo Fuel Tools 9.0.1 (2024-03-14)

  1. Tidy nested namespaces

  2. Update CI badges in README

  3. Create directories and more output on fail

  4. Disable tests that are known to fail on Windows

  5. Update github action workflows

  6. Re-enabling Windows tests

Gazebo Fuel Tools 9.0.0 (2023-09-29)

  1. Added script to update assets to gz

  2. Documentation fixes

  3. ign -> gz

  4. Remove GZ_FUEL_INITIAL_CONFIG_PATH macro

  5. Infrastructure

  6. Update package dependencies for harmonic (gz-msgs10)

  7. ⬆️ Bump main to 9.0.0~pre1

Gazebo Fuel Tools 8.x

Gazebo Fuel Tools 8.1.0 (2023-08-21)

  1. Use pull_request_target for triage workflow

  2. Fix Github project automation for new project board

  3. Zip: use non-deprecated methods

  4. Deprecated non-relocatable macro

  5. Support for bazel in Garden

  6. Minor cleanup

  7. 7.3.0 Release

  8. Add bash completion

  9. Reflect pagination of RESTful requests in iterator API

  10. Disable all LocalCache Tests for Windows

  11. Support link referral download

Gazebo Fuel Tools 8.0.2 (2023-03-30)

  1. Support link referral download

  2. Allow Curl redirect on HTTP requests.

  3. CI and license updates.

  4. Change backpack model version 2->3.

  5. Fix model downloads for ignitionrobotics.org URIs.

  6. Fix build when JSON_USE_EXCEPTION == 0

Gazebo Fuel Tools 8.0.1

  1. Forward ports.

  2. Remove fuel.ignitionrobotics.org from ClientConfig.

  3. Clean up a few Windows warnings.

Gazebo Fuel Tools 8.0.0

  1. Update test infrastruture

  2. Ignition to Gazebo rename.

  3. Bumps in versions for Garden and removal of deprecations

  4. Improve documentation

Gazebo Fuel Tools 7.x

Gazebo Fuel Tools 7.3.0 (2023-06-13)

  1. Forward merges

  2. Add bash completion

  3. Reflect pagination of RESTful requests in iterator API

  4. Support link referral download

Gazebo Fuel Tools 7.2.2 (2023-03-29)

  1. Support link referral download

Gazebo Fuel Tools 7.2.1 (2023-03-22)

  1. Allow Curl redirect on HTTP requests.

  2. CI and license updates.

  3. Change backpack model version 2->3.

Gazebo Fuel Tools 7.2.0 (2021-11-17)

  1. Forward port 4.7.0.

Gazebo Fuel Tools 7.1.0 (2021-08-16)

  1. Ignition to Gazebo server rename effort

  2. Remove redundant namespace references

  3. Change IGN_DESIGNATION to GZ_DESIGNATION

  4. Ignition -> Gazebo docs

  5. Remove unused files (bitbucket-pipelines, docs)

  6. Disable failing windows tests

  7. Disable staging-fuel.ignitionrobotics.org test

  8. Add missing header guard for Interface.hh

  9. SDF from path helper

  10. Add Ubuntu Jammy CI

  11. Fix trailing slashes in failing tests

  12. Print error when unzipping fails to save a file

  13. More checks when manipulating file system in tests

  14. Move test cache to build folder

  15. Improve and fix ResultType tests

  16. Fix cmdfuel ruby script generation on Windows with MSVC

  17. FuelClient.cc: include <deque>

  18. APIs for retrieving models in parallel

Gazebo Fuel Tools 7.0.0 (2021-09-28)

  1. Depend on gz-msgs8

  2. Infrastructure

Gazebo Fuel Tools 6.x

Gazebo Fuel Tools 6.2.0 (2022-03-25)

  1. Fix trailing slashes in failing tests

  2. Print error when unzipping fails to save a file

  3. More checks when manipulating file system in tests

  4. Move test cache to build folder

  5. Improve and fix ResultType tests

  6. FuelClient.cc: include <deque>

  7. APIs for retrieving models in parallel

Gazebo Fuel Tools 6.1.0 (2021-10-15)

  1. Detect gz instead of using cmake module to check for gz-tools

  2. Added fuel update command

  3. Fixed windows download

  4. Owner upload

  5. Infrastructure

Gazebo Fuel Tools 6.0.0 (2021-03-30)

  1. Depend on gz-common4

  2. Depend on gz-msgs7

  3. Unset DELETE method on Windows

  4. Edifice deprecations

  5. Include windows instructions in examples README.md

Gazebo Fuel Tools 5.x

Gazebo Fuel Tools 5.2.0 (2021-12-20)

  1. FuelClient.cc: include <deque>

  2. APIs for retrieving models in parallel

  3. Detect gz instead of using cmake module to check for gz-tools

  4. Added gz fuel update command

  5. Fixed Windows download and CI

  6. Choose which owner to upload to

  7. Included private-token to config.yaml

  8. Add Windows Installation instructions

  9. Use semantic version and prevent crash if version is missing

  10. Document GZ_FUEL_CACHE_PATH on command line

  11. Infrastructure

Gazebo Fuel Tools 5.1.1 (2020-12-18)

  1. Fix light map URI in materials

Gazebo Fuel Tools 5.1.0 (2020-12-09)

  1. Includes all changes up to version 4.3.0.

Gazebo Fuel Tools 5.0.0 (2020-09-28)

  1. Added support for annotation parsing during model creation.

  2. Removed repeated installation instructions.

  3. Update codeowners.

  4. Hide Private headers and LocalCache.hh

  5. Fixed test - Download world 2

Gazebo Fuel Tools 4.x

Gazebo Fuel Tools 4.8.3 (2023-03-29)

  1. Support link referral download

Gazebo Fuel Tools 4.9.1 (2024-01-05)

  1. Create directories and more output on fail

  2. Update github action workflows

  3. Zip: use non-deprecated methods

Gazebo Fuel Tools 4.9.0 (2023-05-03)

  1. Add bash completion

  2. Reflect pagination of RESTful requests in iterator API

Gazebo Fuel Tools 4.8.3 (2023-03-29)

  1. Support link referral download

Gazebo Fuel Tools 4.8.2 (2023-03-21)

  1. Allow Curl redirect HTTP requests.

  2. CI and license updates.

Gazebo Fuel Tools 4.8.1 (2023-02-07)

  1. Fix model downloads for ignitionrobotics.org URIs.

Gazebo Fuel Tools 4.8.0 (2022-12-07)

  1. Reduce asset download sizes.

  2. Update tutorials to use Gazebo.

  3. Change ignitionrobotics.org to gazebosim.org.

Gazebo Fuel Tools 4.7.0 (2022-11-17)

  1. ign -> gz Migrate Ignition Headers : gz-fuel-tools.

Gazebo Fuel Tools 4.6.0 (2022-08-15)

. Ignition to Gazebo server rename * Pull request #263

  1. Remove redundant namespace references

  2. Change IGN_DESIGNATION to GZ_DESIGNATION

  3. Ignition -> Gazebo

  4. Remove unused files (bitbucket-pipelines, docs)

  5. Disable staging-fuel.gazebosim.org test

Gazebo Fuel Tools 4.5.0 (2022-06-01)

  1. APIs for retrieving models in parallel

  2. Add missing header guard for Interface.hh

  3. Print error when unzipping fails to save a file

  4. More checks when manipulating file system in tests

  5. Move test cache to build folder

  6. Improve and fix ResultType tests

  7. FuelClient.cc: include

Gazebo Fuel Tools 4.4.0 (2021-06-17)

  1. Remove tools/code_check and update codecov

  2. Fixed windows download

  3. Master branch updates

  4. Support private-token in config.yaml

  5. Add Windows Installation

  6. Use semantic version and prevent crash if version is missing

  7. Document GZ_FUEL_CACHE_PATH on command line

  8. Support editing/patching model files

  9. Set keep alive on

Gazebo Fuel Tools 4.3.0 (2020-12-01)

  1. Improve fork experience.

  2. Download dependencies.

  3. Includes all changes up to version 3.5.0.

Gazebo Fuel Tools 4.2.1 (2020-08-26)

  1. Fix gz fuel download, which was missing the -j option.

Gazebo Fuel Tools 4.2.0 (2020-08-26)

  1. Set license information based on licenses available from a Fuel server and legal information in a metadata.pbtxt file.

  2. Added edit subcommand to the gz fuel. The edit command currently supports editing a model’s privacy.

  3. Alphabetical listing of subcommands.

  4. All changes up to and including Gazebo Fuel Tools 3.3.0 are included.

Gazebo Fuel Tools 4.1.0 (2020-02-27)

  1. Resource deletion CLI.

  2. Fetch files

  3. README and tutorial updates:

Gazebo Fuel Tools 4.0.0 (2019-12-10)

  1. Model upload CLI, resource metadata CLI, depend on gz-msgs5.

Gazebo Fuel Tools 3.x

Gazebo Fuel Tools 3.x.x (20xx-xx-xx)

Gazebo Fuel Tools 3.5.0 (2020-11-30)

  1. Fix windows build.

  2. Resolve updated codecheck issues.

  3. Update Backpack model to fix tests.

  4. Use lowercase resource and owner names when storing assets on disk.

Gazebo Fuel Tools 3.4.0 (2020-08-19)

  1. Modernize github actions by updating to the new style.

  2. Fix test worlds.

  3. Add capability for downloading collections.

  4. Fix windows build.t

Gazebo Fuel Tools 3.3.0 (2020-07-29)

  1. openrobotics to OpenRobotics

  2. Fix world tests

  3. Add missing dependency in Actions CI

  4. Backport file fetching

  5. Convert model:// to Fuel URLs instead of absolute paths

Gazebo Fuel Tools 3.2.2 (2020-05-18)

  1. Fix URL encodings in RestClient.

  2. Print message when downloading a resource.

  3. Use ${YAML_TARGET} instead of YAML::YAML imported target.

  4. Convert emissive map file path.

  5. Fix windows build with popen and pclose macros.

  6. Remove std::experimental for filesystem (support for VS2019).

Gazebo Fuel Tools 3.2.1 (2019-08-12)

  1. Support actors

Gazebo Fuel Tools 3.2.0 (2019-06-14)

  1. FuelClient::DownloadModel accepts HTTP headers, and the gz fuel download command can accept a single HTTP header.

Gazebo Fuel Tools 3.1.0 (2019-05-xx)

  1. Fix PBR material URI

Gazebo Fuel Tools 3.0.0 (2018-02-28)

  1. Integration with gz-sim. The following set of pull requests update fuel-tools to use gz-cmake2, sets api.ignitionfuel.org as the default server, fixes codecheck errors, removes old deprecations, and parses model.config files using tinyxml2.

Gazebo Fuel Tools 1.x

Gazebo Fuel Tools 1.x.x (20xx-xx-xx)

  1. Use ${YAML_TARGET} instead of YAML::YAML imported target.

Gazebo Fuel Tools 1.2.0 (2018-05-30)

  1. Get cached model resource file

  2. Add some AsString functions

  3. Change cache directory structure and use it

  4. Client return only relevant cached models

  5. Model version support

  6. Remove server API version from model unique name

  7. Client handle unique names and get server info from config

  8. Changed model list to use headers for paging instead of iterating until a 404 is hit. Also added a mechanism to set the user agent

  9. FuelClient: Don’t use ServerConfig if there’s ModelId

  10. List models from command line

  11. Get cached model resource file

  12. Added const constructors for ModelIter and Model

  13. Use common::URI for server URL

  14. Remove server local name

  15. Encode url path before downloading models

  16. Download model from command line

  17. Backport code style changes

Gazebo Fuel Tools 1.0.0 (2018-01-25)