Writes messages to a Postgres database. More...

Inheritance diagram for PostgresSink:

Public Member Functions
	__init__ (self, args=None, **kwargs)

	autodetect (cls, target)
	Returns true if target is recognizable as a Postgres connection string.

	validate (self)
	Returns whether Postgres driver is available, and "commit-interval" and "nesting" in args.write_options have valid value, if any, and database is connectable.

Public Member Functions inherited from BaseDataSink
	__init__ (self, args=None, **kwargs)

	close (self)
	Closes database connection, if any, emits metainfo.

	close_output (self)
	Closes database connection, if any, executing any pending statements.

	emit (self, topic, msg, stamp=None, match=None, index=None)
	Writes message to database.

	validate (self)
	Returns whether args.write_options has valid values, if any.

Public Member Functions inherited from Sink
	__init__ (self, args=None, **kwargs)

	__enter__ (self)
	Context manager entry.

	__exit__ (self, exc_type, exc_value, traceback)
	Context manager exit, closes sink.

	autodetect (cls, target)
	Returns true if target is recognizable as output for this sink class.

	bind (self, source)
	Attaches source to sink.

	close (self)
	Shuts down output, closing any files or connections.

	configure (self, args=None, **kwargs)
	Updates sink configuration.

	emit (self, topic, msg, stamp=None, match=None, index=None)
	Outputs ROS message.

	emit_meta (self)
	Outputs source metainfo like bag header as debug stream, if not already emitted.

	flush (self)
	Writes out any pending data to disk.

	is_highlighting (self)
	Returns whether this sink requires highlighted matches.

	thread_excepthook (self, text, exc)
	Handles exception, used by background threads.

	validate (self)
	Returns whether sink prerequisites are met (like ROS environment set if LiveSink).

	__init__ (self, args=None, **kwargs)

	close (self)
	Clears data structures.

	validate (self)
	Returns whether arguments are valid.

Public Attributes
	ID_SEQUENCE_STEP

	SELECT_TYPE_COLUMNS

	valid

Public Attributes inherited from BaseDataSink
	close

	COMMIT_INTERVAL

	db
	Database connection.

Public Attributes inherited from Sink
	args

	source
	inputs.Source instance bound to this sink

	valid
	Result of validate()

Public Attributes inherited from SqlMixin
	DIALECTS

Static Public Attributes
str	ENGINE = "Postgres"
	Database engine name.

int	ID_SEQUENCE_STEP = 100
	Sequence length per table to reserve for inserted message IDs.

list	MESSAGE_TYPE_BASECOLS
	Default columns for pkg/MsgType tables.

list	MESSAGE_TYPE_NESTCOLS
	Additional default columns for pkg/MsgType tables with nesting output.

list	MESSAGE_TYPE_TOPICCOLS
	Default topic-related columns for pkg/MsgType tables, as [(name, SQL type)].

str	SELECT_TYPE_COLUMNS
	SQL statement for selecting metainfo on pkg/MsgType table columns.

Static Public Attributes inherited from BaseDataSink
int	COMMIT_INTERVAL = 1000
	Number of emits between commits; 0 is autocommit.

	DEFAULT_ARGS = dict(META=False, WRITE_OPTIONS={}, VERBOSE=False)
	Constructor argument defaults.

	ENGINE = None
	Database engine name, overridden in subclasses.

list	MESSAGE_TYPE_BASECOLS
	Default columns for pkg/MsgType tables.

list	MESSAGE_TYPE_NESTCOLS
	Additional default columns for pkg/MsgType tables with nesting output.

list	MESSAGE_TYPE_TOPICCOLS
	Default topic-related columns for pkg/MsgType tables.

Static Public Attributes inherited from Sink
	DEFAULT_ARGS = dict(META=False)
	Constructor argument defaults.

tuple	FILE_EXTENSIONS = ()
	Auto-detection file extensions for subclasses, as (".ext", )

Static Public Attributes inherited from SqlMixin
	DEFAULT_ARGS = dict(WRITE_OPTIONS={})
	Constructor argument defaults.

str	DEFAULT_DIALECT = "sqlite"
	Default SQL dialect used if dialect not specified.

dict	DIALECTS
	Supported SQL dialects and options.

list	KEYWORDS
	Words that need quoting if in name context, like table name.

Detailed Description

Writes messages to a Postgres database.

Output will have:

table "topics", with topic and message type names
table "types", with message type definitions

plus:

table "pkg/MsgType" for each topic message type, with detailed fields, BYTEA fields for uint8[], array fields for scalar list attributes, and JSONB fields for lists of ROS messages; with foreign keys if nesting else subtype values as JSON dictionaries; plus underscore-prefixed fields for metadata, like _topic as the topic name. If not nesting, only topic message type tables are created.
view "/full/topic/name" for each topic, selecting from the message type table

If a message type table already exists but for a type with a different MD5 hash, the new table will have its MD5 hash appended to end, as "pkg/MsgType (hash)".

Definition at line 49 of file postgres.py.

Constructor & Destructor Documentation

◆ init()

__init__	(		self,
			args = `None`,
		**	kwargs
	)

Parameters

args	arguments as namespace or dictionary, case-insensitive; or a single item as the database connection string
args.write	Postgres connection string like "postgresql://user@host/db"
args.write_options	{"commit-interval": transaction size (0 is autocommit), "nesting": "array" to recursively insert arrays of nested types, or "all" for any nesting)}
args.meta	whether to emit metainfo
args.verbose	whether to emit debug information
kwargs	any and all arguments as keyword overrides, case-insensitive

Reimplemented from BaseDataSink.

Definition at line 93 of file postgres.py.

Member Function Documentation

◆ autodetect()

autodetect	(	cls,
		target
	)

Returns true if target is recognizable as a Postgres connection string.

Reimplemented from Sink.

Definition at line 230 of file postgres.py.

◆ validate()

validate ( self )

Returns whether Postgres driver is available, and "commit-interval" and "nesting" in args.write_options have valid value, if any, and database is connectable.

Reimplemented from BaseDataSink.

Definition at line 103 of file postgres.py.

Member Data Documentation

◆ ENGINE

str ENGINE = "Postgres"

static

Database engine name.

Definition at line 52 of file postgres.py.

◆ ID_SEQUENCE_STEP [1/2]

int ID_SEQUENCE_STEP = 100

static

Sequence length per table to reserve for inserted message IDs.

Definition at line 55 of file postgres.py.

◆ ID_SEQUENCE_STEP [2/2]

ID_SEQUENCE_STEP

Definition at line 181 of file postgres.py.

◆ MESSAGE_TYPE_BASECOLS

list MESSAGE_TYPE_BASECOLS

static

Initial value:

=  [("_dt",          "TIMESTAMP"),
                              ("_timestamp",   "NUMERIC"),
                              ("_id",          "BIGSERIAL PRIMARY KEY"), ]

Default columns for pkg/MsgType tables.

Definition at line 71 of file postgres.py.

◆ MESSAGE_TYPE_NESTCOLS

list MESSAGE_TYPE_NESTCOLS

static

Initial value:

= [("_parent_type", "TEXT"),

("_parent_id", "BIGINT"), ]

Additional default columns for pkg/MsgType tables with nesting output.

Definition at line 75 of file postgres.py.

◆ MESSAGE_TYPE_TOPICCOLS

list MESSAGE_TYPE_TOPICCOLS

static

Initial value:

= [("_topic", "TEXT"),

("_topic_id", "BIGINT"), ]

Default topic-related columns for pkg/MsgType tables, as [(name, SQL type)].

Definition at line 68 of file postgres.py.

◆ SELECT_TYPE_COLUMNS [1/2]

str SELECT_TYPE_COLUMNS

static

Initial value:

=  """
    SELECT c.table_name, c.column_name, c.data_type
    FROM   information_schema.columns c INNER JOIN information_schema.tables t
    ON     t.table_name = c.table_name
    WHERE  c.table_schema = current_schema() AND t.table_type = 'BASE TABLE' AND
           c.table_name LIKE '%/%'
    ORDER BY c.table_name, CAST(c.dtd_identifier AS INTEGER)
    """

SQL statement for selecting metainfo on pkg/MsgType table columns.

Definition at line 58 of file postgres.py.

◆ SELECT_TYPE_COLUMNS [2/2]

SELECT_TYPE_COLUMNS

Definition at line 132 of file postgres.py.

◆ valid

valid

Definition at line 113 of file postgres.py.

The documentation for this class was generated from the following file:

grepros/plugins/auto/postgres.py

Public Member Functions

Public Attributes

Static Public Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ autodetect()

◆ validate()

Member Data Documentation

◆ ENGINE

◆ ID_SEQUENCE_STEP [1/2]

◆ ID_SEQUENCE_STEP [2/2]

◆ MESSAGE_TYPE_BASECOLS

◆ MESSAGE_TYPE_NESTCOLS

◆ MESSAGE_TYPE_TOPICCOLS

◆ SELECT_TYPE_COLUMNS [1/2]

◆ SELECT_TYPE_COLUMNS [2/2]

◆ valid

◆ init()