Python package for the evaluation of odometry and SLAM
This packages provides executables and a small library for handling, evaluating and comparing the trajectory output of odometry and SLAM algorithms.
Supported trajectory formats:
- ‘TUM’ trajectory files
- ‘KITTI’ pose files
- ‘EuRoC MAV’ (.csv groundtruth and TUM trajectory file)
- ROS bagfile with geometry_msgs/PoseStamped topics
See here for more infos about the formats.
evo has several advantages over other public benchmarking tools:
- common tools for different formats
- algorithmic options for association, alignment, scale adjustment for monocular SLAM etc.
- flexible options for output, plot or export (e.g. LaTeX plots or Excel tables)
- a powerful, configurable CLI that can cover many use cases
toolslibraries for custom extensions
- faster than other established Python-based tools (see here)
What it’s not: a 1-to-1 re-implementation of a particular evaluation protocol tailored to a specific dataset.
Python 3.4+ and Python 2.7 are both supported. If you want to use the ROS bagfile interface, first check which Python version is used by your ROS installation and install accordingly. You might also want to use a virtual environment.
If you just want to use the executables of the latest release version, the easiest way is to run:
pip install evo --upgrade --no-binary evo
This will download the package and its dependencies from PyPi and install them. Depending on your OS, you might be able to use
pip3 to specify the Python version you want. Tab completion for Bash terminals is supported via the argcomplete package on most UNIX systems - open a new shell after the installation to use it.
Run this in the repository’s base folder:
pip install . --upgrade
evo has the following dependencies that are automatically resolved during installation:
numpy, matplotlib, scipy, pandas, seaborn, natsort, argcomplete, colorama, pygments, enum34 (only Python 2.7)
It is optional but recommended to install PyQt4 before installation, which will give you the enhanced editing tools for plot figures from the “Qt4Agg” matplotlib backend (otherwise: “TkAgg”). If PyQt4 is already installed when installing this package, it will be used as a default. To change the plot backend afterwards, run
evo_config set plot_backend Qt4Agg.
To load or export ROS bag files, you need to install ROS - see here. We tested this package with ROS Indigo and Kinetic.
Command Line Interface
After installation with setup.py or from pip, the following executables can be called globally from your command-line:
evo_ape- absolute pose error
evo_rpe- relative pose error
evo_rpe-for-each- sub-sequence-wise averaged relative pose error
evo_traj- tool for analyzing, plotting or exporting one or more trajectories
evo_res- tool for comparing one or multiple result files from
evo_fig- (experimental) tool for re-opening serialized plots (saved with
evo_config- tool for global settings and config file manipulation
Call the commands with
--help to see the options, e.g.
evo_ape --help. Tab-completion of command line parameters is available on UNIX-like systems.
More documentation Check out the Wiki on GitHub.
There are some example trajectories in the source folder in
- Plot multiple trajectories
Here, we plot two KITTI pose files and the ground truth using
cd test/data evo_traj kitti KITTI_00_ORB.txt KITTI_00_SPTAM.txt --ref=KITTI_00_gt.txt -p --plot_mode=xz
- Run a metric on trajectories
For example, here we calculate the absolute pose error for two trajectories from ORB-SLAM and S-PTAM using
KITTI_00_gt.txt is the reference (ground truth)) and plot and save the individual results to .zip files for
First trajectory (ORB Stereo):
mkdir results evo_ape kitti KITTI_00_gt.txt KITTI_00_ORB.txt -va --plot --save_results results/ORB.zip
Second trajectory (S-PTAM):
evo_ape kitti KITTI_00_gt.txt KITTI_00_SPTAM.txt -va --plot --save_results results/SPTAM.zip
- Process multiple results from a metric
evo_res can be used to compare multiple result files from the metrics, i.e.:
- print infos and statistics (default)
- plot the results
- save the statistics in a table
Here, we use the results from above to generate a plot and a table:
evo_res results/*.zip -p --save_table results/table.csv
IPython / Jupyter Resources
For an interactive source code documentation, open the Jupyter notebook
metrics_tutorial.ipynb in the
notebooks folder of the repository. More infos on Jupyter notebooks: see here
If you have IPython installed, you can launch an IPython shell with a custom evo profile with the command
“:scream:, this piece of :shit: software doesn’t do what I want!!1!1!!”
Jupyter notebook errors
No module named 'evo'
This can be caused if the Kernel version of Jupyter does not match the Python version of the evo installation.
Patches are welcome, preferably as pull requests.
Free, modifiable open source software as covered by the GNU GPL v3 - see the ‘LICENSE’ file for full information. If you use this package for your research, a footnote with the link to this repository is appreciated: