Bulk operations¶
Bulk operations are helpful when you need to create or update several documents within one
Database
with a low amount of requests. In particular, the
_bulk_docs endpoint allows to write a bunch of documents
in one request.
Bulk operations in aiocouch are similar to transactions. You define the set of affected
Document
, apply the changes and finally perform the bulk
request. Depending on the particular task, you need to use one of two context manager
classes.
For example, the following code affects the documents foo and baz, existing or not, and sets the key llama to awesome with one bulk request.
async with database.update_docs(["foo", "baz"], create=True) as bulk:
async for doc in bulk:
doc["llama"] = "awesome"
Include documents in bulk operations¶
Affected documents can be defined in two ways. The first way is to pass a list of document ids as the ids parameter.
async with database.update_docs(ids=["foo", "baz"]) as bulk:
...
The second method is the usage of the append()
method.
Just pass an instance of Document
and its content will be
saved as part of the bulk operation.
the_document = Document(...)
async with BulkOperation(database=my_database) as bulk:
bulk.append(the_document)
Once the control flow leaves the context, the bulk operation persists the applied changes to all documents that there included in the bulk operation one or the other way. Also, both ways can be mixed.
Create many documents in one operation¶
To create many documents, you use the create_docs()
method to get the context manager. Include documents as described above. Once the context
manager closes, one request containing all document contents gets send to the server.
async with my_database.create_docs(...) as bulk:
for doc in bulk:
# make changes to the Document instances
# the request was send now
Note that the bulk operation does not check, if the requested documents alrady exists on
the server. Instead, the error
list will contain
conflict in the error field corresponding to the document.
Update many documents in one operation¶
To update many documents, you use the update_docs()
method to get the context manager. Include documents as described above. Once the context
manager closes, one request containing all document contents gets send to the server. In
contrast to the create operation, the BulkUpdateOperation
context
manager will request all documents whose ids where passed as the ids parameter. If you
already have Document
instance, you may want to use the
append()
method instead.
my_doc: Document = ...
async with my_database.update_docs(...) as bulk:
bulk.append(my_doc)
for doc in docs:
# make changes to the Document instances
# the request was send now
Error handling for bulk operations¶
The important bit first, none of the bulk operation context manager will raise an exception if something in the request went wrong. Each individual document can be saved successfully or may have an error. It’s in your responsibility to check the status after the request finished.
You can check the status of each document with the
ok
, error
, and
response
properties of the context manager. The
ok
and error
lists contain all documents that could and couldn’t be saved properly, respectively. The
response
contains the response from the CouchDB
server. So in case of an error, it will contain a description of what went wrong.
async with BulkOperation(database=my_database) as bulk:
...
if len(bulk.error) == 0:
print(f"Saved all {len(bulk.ok)} documents")
else:
print(f"Failed to saved {len(bulk.error)} documents")
Reference¶
- class aiocouch.bulk.BulkOperation(database)¶
A context manager for bulk operation. This operation allows to write many documents in one request.
Bulk operations use the _bulk_docs endpoint of the database.
To populate the list of written documents, use the
append()
method.- Parameters
database (
Database
) – The database used in the bulk operation
- async for ... in __aiter__()¶
An iterator that yields Document instances that are part of this bulk operation.
- append(doc)¶
Add a document to this bulk operation.
- Parameters
doc (
Document
) – the document that should be stored as part of the bulk operation- Raises
ValueError – if the provided document instance is already part of the bulk operation
- Return type
- Returns
the provided document
- create(id, data=None)¶
Create a new document as part of the bulk operation
- Parameters
- Raises
ValueError – if the provided document id is already part of the bulk operation
- Return type
- Returns
a Document instance reference the newly created document
-
error:
Optional
[List
[Document
]]¶ The list of all
Document
instances that could not be saved to server.Only available after the context manager has finished without a passing exception.
-
ok:
Optional
[List
[Document
]]¶ The list of all
Document
instances that there successfully saved to server.Only available after the context manager has finished without a passing exception.
-
response:
Optional
[List
[Dict
[str
,Any
]]]¶ The resulting JSON response of the _bulk_docs request. Refer to the CouchDB documentation for the contents.
Only available after the context manager has finished without a passing exception.
- property status: Optional[List[Dict[str, Any]]]¶
Deprecated since version 2.1.0: Use the response property instead.
- update(doc)¶
Add a document to this bulk operation.
- Parameters
doc (
Document
) – the document that should be stored as part of the bulk operation- Raises
ValueError – if the provided document instance is already part of the bulk operation
- Return type
- Returns
the provided document
Deprecated since version 2.1.0: Use append(doc) instead. It just makes more sense.
- class aiocouch.bulk.BulkCreateOperation(database, ids=[])¶
A context manager for bulk creation operations. This operation allows to write many documents in one request.
Bulk operations use the _bulk_docs endpoint of the database.
- Parameters
-
error:
Optional
[List
[Document
]]¶ The list of all
Document
instances that could not be saved to server.Only available after the context manager has finished without a passing exception.
- class aiocouch.bulk.BulkUpdateOperation(database, ids=[], create=False)¶
A context manager for bulk update of documents. In particular, for every provided id, a
Document
instance is provided. The data is fetched using theAllDocsView
with a minimal amount of requests.- Parameters
-
error:
Optional
[List
[Document
]]¶ The list of all
Document
instances that could not be saved to server.Only available after the context manager has finished without a passing exception.