mongomotor package

Submodules

mongomotor.connection module

mongomotor.connection.connect(db=None, alias='default', **kwargs)

Connect to the database specified by the ‘db’ argument.

Connection settings may be provided here as well if the database is not running on the default port on localhost. If authentication is needed, provide username and password arguments as well.

Multiple databases are supported by using aliases. Provide a separate alias to connect to a different instance of mongod.

Parameters are the same as for mongoengine.connection.connect() plus one:

Parameters:

async_framework – Which asynchronous framework should be used. It can be tornado or asyncio. Defaults to asyncio.

mongomotor.connection.disconnect(alias='default')

Close the connection with a given alias.

mongomotor.connection.get_db_version(alias='default')

Returns the version of the database for a given alias. This will patch the original mongoengine’s get_mongodb_version.

Parameters:

alias – The alias identifying the connection.

mongomotor.connection.get_mongodb_version(alias='default')

Return the version of the connected mongoDB (first 2 digits)

Parameters:

alias – The alias identifying the connection

Returns:

tuple(int, int)

mongomotor.context_managers module

mongomotor.context_managers.no_dereferencing_active_for_class(cls)

mongomotor.decorators module

mongomotor.decorators.aiter_compat(func)

mongomotor.dereference module

class mongomotor.dereference.MongoMotorDeReference

Bases: DeReference

mongomotor.document module

class mongomotor.document.Document(*args, **kwargs)

Bases: NoDerefInitMixin, Document

The base class used for defining the structure and properties of

collections of documents stored in MongoDB. Inherit from this class, and add fields as class attributes to define a document’s structure. Individual documents may then be created by making instances of the Document subclass.

By default, the MongoDB collection used to store documents created using a Document subclass will be the name of the subclass converted to lowercase. A different collection may be specified by providing collection to the meta dictionary in the class definition.

A Document subclass may be itself subclassed, to create a specialised version of the document that will be stored in the same collection. To facilitate this behaviour a _cls field is added to documents (hidden though the MongoEngine interface). To disable this behaviour and remove the dependence on the presence of _cls set allow_inheritance to False in the meta dictionary.

A Document may use a Capped Collection by specifying max_documents and max_size in the meta dictionary. max_documents is the maximum number of documents that is allowed to be stored in the collection, and max_size is the maximum size of the collection in bytes. max_size is rounded up to the next multiple of 256 by MongoDB internally and mongoengine before. Use also a multiple of 256 to avoid confusions. If max_size is not specified and max_documents is, max_size defaults to 10485760 bytes (10MB).

Indexes may be created by specifying indexes in the meta dictionary. The value should be a list of field names or tuples of field names. Index direction may be specified by prefixing the field names with a + or - sign.

Automatic index creation can be enabled by specifying auto_create_index in the meta dictionary. If this is set to True then indexes will be created by MongoMotor.

By default, _cls will be added to the start of every index (that doesn’t contain a list) if allow_inheritance is True. This can be disabled by either setting cls to False on the specific index or by setting index_cls to False on the meta dictionary for the document.

By default, any extra attribute existing in stored data but not declared

in your model will raise a mongoengine.FieldDoesNotExist error. This can be disabled by setting strict to False in the meta dictionary.

async classmethod compare_indexes()

Compares the indexes defined in MongoEngine with the ones existing in the database. Returns any missing/extra indexes.

async delete(signal_kwargs=None, **write_concern)

Delete the Document from the database. This will only take effect if the document has been previously saved.

Parm signal_kwargs:

(optional) kwargs dictionary to be passed to the signal calls.

Parameters:

write_concern – Extra keyword arguments are passed down which will be used as options for the resultant getLastError command. For example, save(..., write_concern={w: 2, fsync: True}, ...) will wait until at least two servers have recorded the write and will force an fsync on the primary server.

classmethod drop_collection()

Drops the entire collection associated with this mongomotor.Document type from the database.

async classmethod ensure_indexes()

Checks the document meta data and ensures all the indexes exist.

Global defaults can be set in the meta - see guide/defining-documents

By default, this will get called automatically upon first interaction with the Document collection (query, save, etc) so unless you disabled auto_create_index, you shouldn’t have to call this manually.

This also gets called upon every call to Document.save if auto_create_index_on_save is set to True

If called multiple times, MongoDB will not re-recreate indexes if they exist already

Note

You can disable automatic index creation by setting auto_create_index to False in the documents meta data

async modify(query={}, **update)

Perform an atomic update of the document in the database and reload the document object using updated version.

Returns True if the document has been updated or False if the document in the database doesn’t match the query.

Note

All unsaved changes that have been made to the document are rejected if the method returns True.

Parameters:

• query – the update will be performed only if the document in the database matches the query

• update – Django-style update keyword arguments

classmethod register_delete_rule(document_cls, field_name, rule)

This method registers the delete rules to apply when removing this object.

async reload(*fields, **kwargs)

Reloads all attributes from the database.

Parameters:

• fields – (optional) args list of fields to reload

• max_depth – (optional) depth of dereferencing to follow

async save(force_insert=False, validate=True, clean=True, write_concern=None, cascade=None, cascade_kwargs=None, _refs=None, save_condition=None, signal_kwargs=None, **kwargs)

Save the Document to the database. If the document already exists, it will be updated, otherwise it will be created. Returns the saved object instance.

Parameters:

• force_insert – only try to create a new document, don’t allow updates of existing documents.

• validate – validates the document; set to False to skip.

• clean – call the document clean method, requires validate to be True.

• write_concern – Extra keyword arguments are passed down to save() OR insert() which will be used as options for the resultant getLastError command. For example, save(..., write_concern={w: 2, fsync: True}, ...) will wait until at least two servers have recorded the write and will force an fsync on the primary server.

• cascade – Sets the flag for cascading saves. You can set a default by setting “cascade” in the document __meta__

• cascade_kwargs – (optional) kwargs dictionary to be passed throw to cascading saves. Implies cascade=True.

• _refs – A list of processed references used in cascading saves

• save_condition – only perform save if matching record in db satisfies condition(s) (e.g. version number). Raises OperationError if the conditions are not satisfied

• signal_kwargs – (optional) kwargs dictionary to be passed to the signal calls.

async select_related(max_depth=1)

Handles dereferencing of DBRef objects or ObjectId a maximum depth in order to cut down the number queries to mongodb.

class mongomotor.document.DynamicDocument(*args, **kwargs)

Bases: Document, DynamicDocument

class mongomotor.document.DynamicEmbeddedDocument(*args, **kwargs)

Bases: NoDerefInitMixin, DynamicEmbeddedDocument

class mongomotor.document.EmbeddedDocument(*args, **kwargs)

Bases: NoDerefInitMixin, EmbeddedDocument

class mongomotor.document.NoDerefInitMixin(*args, **kwargs)

Bases: object

A mixin used to Documents and EmbeddedDocuments not to dereference reference fields on __init__.

mongomotor.exceptions module

exception mongomotor.exceptions.BadAsyncFrameworkError

Bases: Exception

exception mongomotor.exceptions.ConfusionError

Bases: Exception

exception mongomotor.exceptions.MissingFramework

Bases: Exception

mongomotor.fields module

class mongomotor.fields.BaseAsyncReferenceField

Bases: object

Base class for async reference fields.

class mongomotor.fields.ComplexBaseField(field=None, **kwargs)

Bases: ComplexBaseField

class mongomotor.fields.DictField(field=None, *args, **kwargs)

Bases: ComplexBaseField, DictField

class mongomotor.fields.FileField(db_alias='default', collection_name='fs', **kwargs)

Bases: FileField

proxy_class

alias of GridFSProxy

class mongomotor.fields.GenericReferenceField(*args, **kwargs)

Bases: BaseAsyncReferenceField, GenericReferenceField

class mongomotor.fields.GridFSProxy(*args, **kwargs)

Bases: GridFSProxy

async close()

async delete()

property fs

async get(grid_id=None)

async put(file_obj, **kwargs)

async read(size=-1)

async replace(data, **metadata)

Replaces the contents of the file with data.

Parameters:

• data – A byte-string to write to gridfs.

• metatada – File metadata.

async write(data)

Writes data to gridfs.

Parameters:

data – String or bytes to write to gridfs.

class mongomotor.fields.ListField(field=None, *, max_length=None, **kwargs)

Bases: ComplexBaseField, ListField

class mongomotor.fields.ReferenceField(document_type, dbref=False, reverse_delete_rule=0, **kwargs)

Bases: BaseAsyncReferenceField, ReferenceField

A reference to a document that will be dereferenced on access.

Use the reverse_delete_rule to handle what should happen if the document the field is referencing is deleted. EmbeddedDocuments, DictFields and MapFields does not support reverse_delete_rule and an InvalidDocumentError will be raised if trying to set on one of these Document / Field types.

The options are:

• DO_NOTHING (0) - don’t do anything (default).

• NULLIFY (1) - Updates the reference to null.

• CASCADE (2) - Deletes the documents associated with the reference.

• DENY (3) - Prevent the deletion of the reference object.

• PULL (4) - Pull the reference from a ListField of references

Alternative syntax for registering delete rules (useful when implementing bi-directional delete rules)

class Bar(Document):
    content = StringField()
    foo = ReferenceField('Foo')

Foo.register_delete_rule(Bar, 'foo', NULLIFY)

mongomotor.monkey module

class mongomotor.monkey.MonkeyPatcher

Bases: object

patch_async_connections()

Patches mongoengine.connection._connections removing all asynchronous connections from there.

It is used when switching to a synchronous connection to avoid mongoengine returning a asynchronous connection with the same configuration.

patch_db_clients(client)

Patches the db clients used to connect to mongodb.

Parameters:

client – Which client should be used.

patch_dereference()

patch_get_mongodb_version()

Patches mongoengine’s get_mongodb_version to use a function does not reach the database.

patch_item(obj, attr, newitem, undo=True)

Sets attr in obj with newitem. If not undo the item will continue patched after leaving the context manager

patch_no_dereferencing_active_for_class()

patch_qs_stop_iteration()

Patches StopIterations raised by mongoengine’s queryset replacing it by AsyncStopIteration so it can interact well with futures.

patch_signals()

Patches mongoengine signals to use asyncblink signals

patch_sync_connections()

Patches mongoengine.connection._connections removing all synchronous connections from there.

mongomotor.queryset module

class mongomotor.queryset.QuerySet(document, collection)

Bases: QuerySet

async aggregate(pipeline, **kwargs)

Perform an aggregate function based on your queryset params

Parameters:

• pipeline – list of aggregation commands, see: https://www.mongodb.com/docs/manual/core/aggregation-pipeline/

• kwargs – (optional) kwargs dictionary to be passed to pymongo’s aggregate call.

async average(field)

Average over the values of the specified field.

Parameters:

field – the field to average over; use dot-notation to refer to embedded document fields

This method is more performant than the regular average, because it uses the aggregation framework instead of map-reduce.

async count(with_limit_and_skip=True)

Counts the documents in the queryset.

Parameters:

with_limit_and_skip – Indicates if limit and skip applied to the queryset should be taken into account.

async delete(write_concern=None, _from_doc_delete=False, cascade_refs=None)

Deletes the documents matched by the query.

Parameters:

• write_concern – Extra keyword arguments are passed down which will be used as options for the resultant getLastError command. For example, save(..., write_concern={w: 2, fsync: True}, ...) will wait until at least two servers have recorded the write and will force an fsync on the primary server.

• _from_doc_delete – True when called from document delete therefore signals will have been triggered so don’t loop.

:returns number of deleted documents

async distinct(field)

Return a list of distinct values for a given field.

Parameters:

field – the field to select distinct values from

Note

This is a command and won’t take ordering or limit into account.

async explain()

Return an explain plan record for the QuerySet cursor.

property fetch_next

async first()

Retrieve the first object matching the query.

async get(*q_objs, **query)

Retrieve the the matching object raising MultipleObjectsReturned or DocumentName.MultipleObjectsReturned exception if multiple results and DoesNotExist or DocumentName.DoesNotExist if no results are found.

async in_bulk(object_ids)

Retrieve a set of documents by their ids.

Parameters:

object_ids – a list or tuple of ObjectId’s

Return type:

dict of ObjectId’s as keys and collection-specific Document subclasses as values.

async insert(doc_or_docs, load_bulk=True, write_concern=None, signal_kwargs=None)

bulk insert documents

Parameters:

• doc_or_docs – a document or list of documents to be inserted

• (optional) (load_bulk) – If True returns the list of document instances

• write_concern – Extra keyword arguments are passed down to insert() which will be used as options for the resultant getLastError command. For example, insert(..., {w: 2, fsync: True}) will wait until at least two servers have recorded the write and will force an fsync on each server being written to.

• signal_kwargs – (optional) kwargs dictionary to be passed to the signal calls.

By default returns document instances, set load_bulk to False to return just ObjectIds

async item_frequencies(field, normalize=False)

Returns a dictionary of all items present in a field across the whole queried set of documents, and their corresponding frequency. This is useful for generating tag clouds, or searching documents.

Note

Can only do direct simple mappings and cannot map across ReferenceField or GenericReferenceField for more complex counting a manual aggretation call would be required.

If the field is a ListField, the items within each list will be counted individually.

Parameters:

• field – the field to use

• normalize – normalize the results so they add to 1.0

async map_reduce(map_f, reduce_f, output, finalize_f=None, limit=None, scope=None)

Perform a map/reduce query using the current query spec and ordering. While map_reduce respects QuerySet chaining, it must be the last call made, as it does not return a maleable QuerySet.

See the test_map_reduce() and test_map_advanced() tests in tests.queryset.QuerySetTest for usage examples.

Parameters:

• map_f – map function, as Code or string

• reduce_f – reduce function, as Code or string

• output – output collection name, if set to ‘inline’ will return the results inline. This can also be a dictionary containing output options see: https://www.mongodb.com/docs/manual/reference/command/mapReduce/#mongodb-dbcommand-dbcmd.mapReduce

• finalize_f – finalize function, an optional function that performs any post-reduction processing.

• scope – values to insert into map/reduce global scope. Optional.

• limit – number of objects from current query to provide to map/reduce method

Returns an iterator yielding MapReduceDocument.

async modify(upsert=False, remove=False, new=False, array_filters=None, **update)

Update and return the updated document.

Returns either the document before or after modification based on new parameter. If no documents match the query and upsert is false, returns None. If upserting and new is false, returns None.

Parameters:

• upsert – insert if document doesn’t exist (default False)

• full_response – return the entire response object from the server (default False, not available for PyMongo 3+)

• remove – remove rather than updating (default False)

• new – return updated rather than original document (default False)

• array_filters – A list of filters specifying which array elements an update should apply.

• update – Django-style update keyword arguments

next_object()

no_cache()

Convert to a non-caching queryset

async sum(field)

Sum over the values of the specified field.

Parameters:

field – the field to sum over; use dot-notation to refer to embedded document fields

This method is more performant than the regular sum, because it uses the aggregation framework instead of map-reduce.

async to_list(length=100)

Returns a list of the current documents in the queryset.

Parameters:

length – maximum number of documents to return for this call.

async update(upsert=False, multi=True, write_concern=None, read_concern=None, full_result=False, array_filters=None, **update)

Perform an atomic update on the fields matched by the query.

Parameters:

• upsert – insert if document doesn’t exist (default False)

• multi – Update multiple documents.

• write_concern – Extra keyword arguments are passed down which will be used as options for the resultant getLastError command. For example, save(..., write_concern={w: 2, fsync: True}, ...) will wait until at least two servers have recorded the write and will force an fsync on the primary server.

• read_concern – Override the read concern for the operation

• full_result – Return the associated pymongo.UpdateResult rather than just the number updated items

• array_filters – A list of filters specifying which array elements an update should apply.

• update – Django-style update keyword arguments

:returns the number of updated documents (unless full_result

is True)

async upsert_one(write_concern=None, **update)

Overwrite or add the first document matched by the query.

Parameters:

• write_concern – Extra keyword arguments are passed down which will be used as options for the resultant getLastError command. For example, save(..., write_concern={w: 2, fsync: True}, ...) will wait until at least two servers have recorded the write and will force an fsync on the primary server.

• update – Django-style update keyword arguments

:returns the new or overwritten document

class mongomotor.queryset.QuerySetNoCache(document, collection)

Bases: QuerySet

A non caching QuerySet

cache()

Convert to a caching queryset

mongomotor.signals module

mongomotor.utils module

mongomotor.utils.get_alias_for_db(db)

Return the alias for a given db.

mongomotor.utils.get_sync_alias(alias)

Returns an alias to be used for the sync connection of alias.

mongomotor.utils.is_main_thread()

Module contents

class mongomotor.Document(*args, **kwargs)

Bases: NoDerefInitMixin, Document

The base class used for defining the structure and properties of

collections of documents stored in MongoDB. Inherit from this class, and add fields as class attributes to define a document’s structure. Individual documents may then be created by making instances of the Document subclass.

By default, the MongoDB collection used to store documents created using a Document subclass will be the name of the subclass converted to lowercase. A different collection may be specified by providing collection to the meta dictionary in the class definition.

A Document subclass may be itself subclassed, to create a specialised version of the document that will be stored in the same collection. To facilitate this behaviour a _cls field is added to documents (hidden though the MongoEngine interface). To disable this behaviour and remove the dependence on the presence of _cls set allow_inheritance to False in the meta dictionary.

A Document may use a Capped Collection by specifying max_documents and max_size in the meta dictionary. max_documents is the maximum number of documents that is allowed to be stored in the collection, and max_size is the maximum size of the collection in bytes. max_size is rounded up to the next multiple of 256 by MongoDB internally and mongoengine before. Use also a multiple of 256 to avoid confusions. If max_size is not specified and max_documents is, max_size defaults to 10485760 bytes (10MB).

Indexes may be created by specifying indexes in the meta dictionary. The value should be a list of field names or tuples of field names. Index direction may be specified by prefixing the field names with a + or - sign.

Automatic index creation can be enabled by specifying auto_create_index in the meta dictionary. If this is set to True then indexes will be created by MongoMotor.

By default, _cls will be added to the start of every index (that doesn’t contain a list) if allow_inheritance is True. This can be disabled by either setting cls to False on the specific index or by setting index_cls to False on the meta dictionary for the document.

By default, any extra attribute existing in stored data but not declared

in your model will raise a mongoengine.FieldDoesNotExist error. This can be disabled by setting strict to False in the meta dictionary.

async classmethod compare_indexes()

Compares the indexes defined in MongoEngine with the ones existing in the database. Returns any missing/extra indexes.

async delete(signal_kwargs=None, **write_concern)

Delete the Document from the database. This will only take effect if the document has been previously saved.

Parm signal_kwargs:

(optional) kwargs dictionary to be passed to the signal calls.

Parameters:

write_concern – Extra keyword arguments are passed down which will be used as options for the resultant getLastError command. For example, save(..., write_concern={w: 2, fsync: True}, ...) will wait until at least two servers have recorded the write and will force an fsync on the primary server.

classmethod drop_collection()

Drops the entire collection associated with this mongomotor.Document type from the database.

async classmethod ensure_indexes()

Checks the document meta data and ensures all the indexes exist.

Global defaults can be set in the meta - see guide/defining-documents

By default, this will get called automatically upon first interaction with the Document collection (query, save, etc) so unless you disabled auto_create_index, you shouldn’t have to call this manually.

This also gets called upon every call to Document.save if auto_create_index_on_save is set to True

If called multiple times, MongoDB will not re-recreate indexes if they exist already

Note

You can disable automatic index creation by setting auto_create_index to False in the documents meta data

async modify(query={}, **update)

Perform an atomic update of the document in the database and reload the document object using updated version.

Returns True if the document has been updated or False if the document in the database doesn’t match the query.

Note

All unsaved changes that have been made to the document are rejected if the method returns True.

Parameters:

• query – the update will be performed only if the document in the database matches the query

• update – Django-style update keyword arguments

classmethod register_delete_rule(document_cls, field_name, rule)

This method registers the delete rules to apply when removing this object.

async reload(*fields, **kwargs)

Reloads all attributes from the database.

Parameters:

• fields – (optional) args list of fields to reload

• max_depth – (optional) depth of dereferencing to follow

async save(force_insert=False, validate=True, clean=True, write_concern=None, cascade=None, cascade_kwargs=None, _refs=None, save_condition=None, signal_kwargs=None, **kwargs)

Save the Document to the database. If the document already exists, it will be updated, otherwise it will be created. Returns the saved object instance.

Parameters:

• force_insert – only try to create a new document, don’t allow updates of existing documents.

• validate – validates the document; set to False to skip.

• clean – call the document clean method, requires validate to be True.

• write_concern – Extra keyword arguments are passed down to save() OR insert() which will be used as options for the resultant getLastError command. For example, save(..., write_concern={w: 2, fsync: True}, ...) will wait until at least two servers have recorded the write and will force an fsync on the primary server.

• cascade – Sets the flag for cascading saves. You can set a default by setting “cascade” in the document __meta__

• cascade_kwargs – (optional) kwargs dictionary to be passed throw to cascading saves. Implies cascade=True.

• _refs – A list of processed references used in cascading saves

• save_condition – only perform save if matching record in db satisfies condition(s) (e.g. version number). Raises OperationError if the conditions are not satisfied

• signal_kwargs – (optional) kwargs dictionary to be passed to the signal calls.

async select_related(max_depth=1)

Handles dereferencing of DBRef objects or ObjectId a maximum depth in order to cut down the number queries to mongodb.

class mongomotor.DynamicDocument(*args, **kwargs)

Bases: Document, DynamicDocument

class mongomotor.DynamicEmbeddedDocument(*args, **kwargs)

Bases: EmbeddedDocument

A Dynamic Embedded Document class allowing flexible, expandable and uncontrolled schemas. See DynamicDocument for more information about dynamic documents.

my_metaclass

alias of DocumentMetaclass

class mongomotor.EmbeddedDocument(*args, **kwargs)

Bases: NoDerefInitMixin, EmbeddedDocument

class mongomotor.MapReduceDocument(document, collection, key, value)

Bases: object

A document returned from a map/reduce query.

Parameters:

• collection – An instance of Collection

• key – Document/result key, often an instance of ObjectId. If supplied as an ObjectId found in the given collection, the object can be accessed via the object property.

• value – The result(s) for this key.

property object

Lazy-load the object referenced by self.key. self.key should be the primary_key.

mongomotor.connect(db=None, alias='default', **kwargs)

Connect to the database specified by the ‘db’ argument.

Connection settings may be provided here as well if the database is not running on the default port on localhost. If authentication is needed, provide username and password arguments as well.

Multiple databases are supported by using aliases. Provide a separate alias to connect to a different instance of mongod.

Parameters are the same as for mongoengine.connection.connect() plus one:

Parameters:

async_framework – Which asynchronous framework should be used. It can be tornado or asyncio. Defaults to asyncio.

mongomotor.disconnect(alias='default')

Close the connection with a given alias.