Create Tailable Cursor

Overview¶

By default, MongoDB will automatically close a cursor when the client has exhausted all results in the cursor. However, for capped collections you may use a Tailable Cursor that remains open after the client exhausts the results in the initial cursor. Tailable cursors are conceptually equivalent to the tail Unix command with the -f option (i.e. with “follow” mode.) After clients insert new additional documents into a capped collection, the tailable cursor will to continue to retrieve documents.

Use tailable cursors on capped collections with high numbers of write operations for which an index would be too expensive. For instance, MongoDB replication uses tailable cursors to tail the primary’s oplog.

db.<collection>.find( { indexedField: { $gt: <lastvalue> } } )

Consider the following behaviors related to tailable cursors:

• Tailable cursors do not use indexes and return documents in natural order.

• Because tailable cursors do not use indexes, the initial scan for the query may be expensive; but, after initially exhausting the cursor, subsequent retrievals of the newly added documents are inexpensive.

• Tailable cursors may become dead, or invalid, if either:

  • the query returns no match.
  • the cursor returns the document at the “end” of the collection and then the application deletes those document.

  A dead cursor has an id of 0.

See your driver documentation for the driver-specific method to specify the tailable cursor. For more information on the details of specifying a tailable cursor, see MongoDB wire protocol documentation.

C++ Example¶

The tail function uses a tailable cursor to output the results from a query to a capped collection:

• The function handles the case of the dead cursor by having the query be inside a loop.
• To periodically check for new data, the cursor->more() statement is also inside a loop.

#include "client/dbclient.h"

using namespace mongo;

/*
 * Example of a tailable cursor.
 * The function "tails" the capped collection (ns) and output elements as they are added.
 * The function also handles the possibility of a dead cursor by tracking the field 'insertDate'.
 * New documents are added with increasing values of 'insertDate'.
 */

void tail(DBClientBase& conn, const char *ns) {

    BSONElement lastValue = minKey.firstElement();

    Query query = Query().hint( BSON( "$natural" << 1 ) );

    while ( 1 ) {
        auto_ptr<DBClientCursor> c =
            conn.query(ns, query, 0, 0, 0,
                       QueryOption_CursorTailable | QueryOption_AwaitData );

        while ( 1 ) {
            if ( !c->more() ) {

                if ( c->isDead() ) {
                    break;
                }

                continue;
            }

            BSONObj o = c->next();
            lastValue = o["insertDate"];
            cout << o.toString() << endl;
        }

        query = QUERY( "insertDate" << GT << lastValue ).hint( BSON( "$natural" << 1 ) );
    }
}

The tail function performs the following actions:

• Initialize the lastValue variable, which tracks the last accessed value. The function will use the lastValue if the cursor becomes invalid and tail needs to restart the query. Use hint() to ensure that the query uses the $natural order.

• In an outer while(1) loop,

  • Query the capped collection and return a tailable cursor that blocks for several seconds waiting for new documents

    auto_ptr<DBClientCursor> c =
         conn.query(ns, query, 0, 0, 0,
                    QueryOption_CursorTailable | QueryOption_AwaitData );
    

    • Specify the capped collection using ns as an argument to the function.
    • Set the QueryOption_CursorTailable option to create a tailable cursor.
    • Set the QueryOption_AwaitData option so that the returned cursor blocks for a few seconds to wait for data.

  • In an inner while (1) loop, read the documents from the cursor:

    • If the cursor has no more documents and is not invalid, loop the inner while loop to recheck for more documents.
    • If the cursor has no more documents and is dead, break the inner while loop.
    • If the cursor has documents:
      • output the document,
      • update the lastValue value,
      • and loop the inner while (1) loop to recheck for more documents.

  • If the logic breaks out of the inner while (1) loop and the cursor is invalid:

    • Use the lastValue value to create a new query condition that matches documents added after the lastValue. Explicitly ensure $natural order with the hint() method:

      query = QUERY( "insertDate" << GT << lastValue ).hint( BSON( "$natural" << 1 ) );
      

    • Loop through the outer while (1) loop to re-query with the new query condition and repeat.