grep for ROS bag files and live topics
Index
Installation
Using the program
Matching and filtering
Outputs
Command-line options
Plugins
Changelog
API documentation
View the Project on GitHub suurjaak/grepros
All command-line arguments:
positional arguments:
PATTERN pattern(s) to find in message field values,
all messages match if not given,
can specify message field as NAME=PATTERN
(supports nested.paths and * wildcards)
optional arguments:
-h, --help show this help message and exit
-F, --fixed-strings PATTERNs are ordinary strings, not regular expressions
-I, --no-ignore-case use case-sensitive matching in PATTERNs
-v, --invert-match select messages not matching PATTERN
--version display version information and exit
--live read messages from live ROS topics instead of bagfiles
--publish publish matched messages to live ROS topics
--write TARGET [format=bag|csv|html|mcap|parquet|postgres|sql|sqlite] [KEY=VALUE ...]
write matched messages to specified output,
format is autodetected from TARGET if not specified.
Bag or database will be appended to if it already exists.
Keyword arguments are given to output writer.
column-NAME=ROSTYPE:VALUE
additional column to add in Parquet output,
like column-bag_hash=string:26dfba2c
commit-interval=NUM transaction size for Postgres/SQLite output
(default 1000, 0 is autocommit)
dialect-file=path/to/dialects.yaml
load additional SQL dialect options
for Postgres/SQL/SQLite output
from a YAML or JSON file
dialect=clickhouse|postgres|sqlite
use specified SQL dialect in SQL output
(default "sqlite")
idgenerator=CALLABLE callable or iterable for producing message IDs
in Parquet output, like 'uuid.uuid4' or 'itertools.count()';
nesting uses UUID values by default
message-yaml=true|false whether to populate table field messages.yaml
in SQLite output (default true)
nesting=array|all create tables for nested message types
in Parquet/Postgres/SQL/SQLite output,
only for arrays if "array"
else for any nested types
(array fields in parent will be populated
with foreign keys instead of messages as JSON)
overwrite=true|false overwrite existing file
in bag/CSV/HTML/MCAP/Parquet/SQL/SQLite output
instead of appending to if bag or database
or appending unique counter to file name
(default false)
rollover-size=NUM size limit for individual files
in bag/HTML/MCAP/SQLite output
as bytes (supports abbreviations like 1K or 2M or 3G)
rollover-count=NUM message limit for individual files
in bag/HTML/MCAP/SQLite output
(supports abbreviations like 1K or 2M or 3G)
rollover-duration=INTERVAL
message time span limit for individual files
in bag/HTML/MCAP/SQLite output
as seconds (supports abbreviations like 60m or 2h or 1d)
rollover-template=STR output filename template for individual files
in bag/HTML/MCAP/SQLite output,
supporting strftime format codes like "%H-%M-%S"
and "%(index)s" as output file index
template=/my/path.tpl custom template to use for HTML output
type-ROSTYPE=ARROWTYPE custom mapping between ROS and pyarrow type
for Parquet output, like type-time="timestamp('ns')"
or type-uint8[]="list(uint8())"
writer-ARGNAME=ARGVALUE additional arguments for Parquet output
given to pyarrow.parquet.ParquetWriter
--plugin PLUGIN [PLUGIN ...]
load a Python module or class as plugin
(built-in plugins: grepros.plugins.embag,
grepros.plugins.mcap, grepros.plugins.parquet,
grepros.plugins.sql)
--stop-on-error stop further execution on any error like unknown message type
Filtering:
-t TOPIC [TOPIC ...], --topic TOPIC [TOPIC ...]
ROS topics to read if not all (supports * wildcards)
-nt TOPIC [TOPIC ...], --no-topic TOPIC [TOPIC ...]
ROS topics to skip (supports * wildcards)
-d TYPE [TYPE ...], --type TYPE [TYPE ...]
ROS message types to read if not all (supports * wildcards)
-nd TYPE [TYPE ...], --no-type TYPE [TYPE ...]
ROS message types to skip (supports * wildcards)
--condition CONDITION [CONDITION ...]
extra conditions to require for matching messages,
as ordinary Python expressions, can refer to last messages
in topics as <topic /my/topic>; topic name can contain wildcards.
E.g. --condition "<topic /robot/enabled>.data" matches
messages only while last message in '/robot/enabled' has data=true.
-t0 TIME, --start-time TIME
earliest timestamp of messages to read
as relative seconds if signed,
or epoch timestamp or ISO datetime
(for bag input, relative to bag start time
if positive or end time if negative,
for live input relative to system time,
datetime may be partial like 2021-10-14T12)
-t1 TIME, --end-time TIME
latest timestamp of messages to read
as relative seconds if signed,
or epoch timestamp or ISO datetime
(for bag input, relative to bag start time
if positive or end time if negative,
for live input relative to system time,
datetime may be partial like 2021-10-14T12)
-n0 INDEX, --start-index INDEX
message index within topic to start from
(1-based if positive, counts back from bag total if negative)
-n1 INDEX, --end-index INDEX
message index within topic to stop at
(1-based if positive, counts back from bag total if negative)
--every-nth-message NUM
read every Nth message within topic
--every-nth-interval SECONDS
read messages at least N seconds apart within topic
--every-nth-match NUM
emit every Nth match in topic
-sf FIELD [FIELD ...], --select-field FIELD [FIELD ...]
message fields to use in matching if not all
(supports nested.paths and * wildcards)
-ns FIELD [FIELD ...], --no-select-field FIELD [FIELD ...]
message fields to skip in matching
(supports nested.paths and * wildcards)
-m NUM, --max-count NUM
number of matched messages to emit (per each file if bag input)
--max-per-topic NUM number of matched messages to emit from each topic
(per each file if bag input)
--max-topics NUM number of topics to emit matches from (per each file if bag input)
--unique-only only emit matches that are unique in topic,
taking --select-field and --no-select-field into account
(per each file if bag input)
Output control:
-B NUM, --before-context NUM
emit NUM messages of leading context before match
-A NUM, --after-context NUM
emit NUM messages of trailing context after match
-C NUM, --context NUM
emit NUM messages of leading and trailing context
around match
-ef FIELD [FIELD ...], --emit-field FIELD [FIELD ...]
message fields to emit in console/CSV/HTML/Parquet output if not all
(supports nested.paths and * wildcards)
-nf FIELD [FIELD ...], --no-emit-field FIELD [FIELD ...]
message fields to skip in console/CSV/HTML/Parquet output
(supports nested.paths and * wildcards)
-mo, --matched-fields-only
emit only the fields where PATTERNs find a match in console/HTML output
-la NUM, --lines-around-match NUM
emit only matched fields and NUM message lines
around match in console/HTML output
-lf NUM, --lines-per-field NUM
maximum number of lines to emit per field in console/HTML output
-l0 NUM, --start-line NUM
message line number to start emitting from in console/HTML output
(1-based if positive, counts back from total if negative)
-l1 NUM, --end-line NUM
message line number to stop emitting at in console/HTML output
(1-based if positive, counts back from total if negative)
-lm NUM, --lines-per-message NUM
maximum number of lines to emit per message in console/HTML output
--match-wrapper [STR [STR ...]]
string to wrap around matched values in console/HTML output,
both sides if one value, start and end if more than one,
or no wrapping if zero values
(default "**" in colorless output)
--wrap-width NUM character width to wrap message YAML console output at,
0 disables (defaults to detected terminal width)
--color {auto,always,never}
use color output in console (default "always")
--no-meta do not print source and message metainfo to console
--no-filename do not print bag filename prefix on each console message line
--no-highlight do not highlight matched values
--no-console-output do not print matches to console
--progress show progress bar when not printing matches to console
--verbose print status messages during console output
for publishing and writing
--no-verbose do not print status messages during console output
for publishing and writing
Bag input control:
-n FILE [FILE ...], --filename FILE [FILE ...]
names of ROS bagfiles to read if not all in directory
(supports * wildcards)
-p PATH [PATH ...], --path PATH [PATH ...]
paths to scan if not current directory
(supports * wildcards)
-r, --recursive recurse into subdirectories when looking for bagfiles
--order-bag-by {topic,type}
order bag messages by topic or type first and then by time
--decompress decompress archived bagfiles with recognized extensions (.zst .zstd)
--reindex-if-unindexed
reindex unindexed bagfiles (ROS1 only), makes backup copies
Live topic control:
--publish-prefix PREFIX
prefix to prepend to input topic name on publishing match
--publish-suffix SUFFIX
suffix to append to input topic name on publishing match
--publish-fixname TOPIC
single output topic name to publish all matches to,
overrides prefix and suffix
--queue-size-in SIZE live ROS topic subscriber queue size (default 10)
--queue-size-out SIZE
output publisher queue size (default 10)
--ros-time-in use ROS time instead of system time for incoming message
timestamps from subsribed live ROS topics
Loaded plugins will add their additional arguments to help output.