Duplicate Key error not detected

[Andrew J Westlake]

Background

At the moment I'm using this library with MongoDB 4.0.8.

In my database, I have a "users" collection where the "username" field is a unique index". I've added this initialization script in order to create the unique index:

db.users.createIndex({ username: 1 }, { unique: true })

Expected behavior

In mongo-express, when I attempt to add a new user with a duplicate username to the collection, it behaves as expected and gives me a duplicate key error.

Actual behavior

When I add the user through this library, it behaves as if the insertion happened, but nothing changes in the database. I think it must just be ignoring this error somewhere along the way.

Is there a reason for this behavior, and can I expect this error to be supported in the near future?
I'm not opposed to implementing the fix and creating a pull request, I just wanted to bring it to your attention first.

[Saghm Rossi]

Hi! Can you post an example of the code you're running in the library that's causing this issue? Additionally, if you're creating the index through the shell and not the driver, can you provide which database you're creating the users in along with that code?

[Andrew J Westlake]

Here's the example code for inserting a user. I'm using r2d2_mongodb to manage my database connections.

use bson::{bson, doc, Bson};
use mongodb::{coll::options::FindOptions, db::ThreadedDatabase};
use r2d2::Pool;
use r2d2_mongodb::MongodbConnectionManager;

use crate::err::{Error, Result};

#[derive(Clone)]
pub struct App {
    pool: Pool<MongodbConnectionManager>,
}

impl App {
    pub fn create_user(
        &self,
        username: String,
        name: Option<String>,
        picture: Option<String>,
    ) -> Result<()> {
        let conn = self.pool.get()?;

        let users = conn.collection("users");
        users.insert_one(
            doc! {
                "username": username,
                "name": name.map(|name| Bson::String(name)).unwrap_or(Bson::Null),
                "picture": picture.map(|pic| Bson::String(pic)).unwrap_or(Bson::Null),
            },
            None,
        )?;

        Ok(())
    }
}

I'm not creating the index through the driver. Instead, I'm using the mongo docker image with an initialization script:

Dockerfile:

FROM mongo:latest

ENV MONGO_INITDB_DATABASE example

COPY ./init-example.js /docker-entrypoint-initdb.d/

Init script:

db.users.createIndex({ username: 1 }, { unique: true })

[Saghm Rossi]

The code you shared using r2d2 doesn't show which database you're using, but the initialization script seems to be using the shell's default database (i.e. test). I suspect that the issue is due to the code using a different database where the user hasn't already been created.

[Saghm Rossi]

It looks like r2d2-mongodb defaults to using admin, which would explain the issue. You can change the database that the initialization script creates the user in by prepending the line use admin to your script, or you can change the database that your connection pool uses by default with this method (or both, of course).

[Andrew J Westlake]

Thanks for responding so quickly! I appreciate the help with this problem.

I don't know if r2d2's defaults are the problem here, though. I should have included the creation of the r2d2 connection pool, because I was overriding the default admin database. I went back and verified that the documents were not being inserted into the admin database.

Here's a full example that I've tested with a fresh database called 'example'. The index creation is fully in rust and the client connects to the same mongodb server that I was using earlier. I even ripped out r2d2 to isolate the issue.

use bson::{bson, doc};
use mongodb::{Client, ThreadedClient, db::{Database, ThreadedDatabase}, coll::options::IndexOptions};

fn create_user(
    database: &Database,
    username: &str,
) {
    let users = database.collection("users");
    users.insert_one(
        doc! {
            "username": username,
        },
        None,
    ).unwrap();
}

fn main() {
    let client = Client::connect("database", 27017).unwrap();
    let database = client.db("example");

    let mut index_options = IndexOptions::new();
    index_options.unique = Some(true);
    database.collection("users").create_index(doc! { "username": 1 }, Some(index_options)).unwrap();

    create_user(&database, "example-user");
    create_user(&database, "example-user");
}

I expect this example to panic with the Duplicate Keys error, but the program runs successfully.

This image shows the "users" collection after the operation. There is only one user here named "example-user" as expected.

This image shows the indexes that were created for the "users" collection after the program runs. There is a unique username index as expected.

I mentioned above that this was a fresh database. I made sure that there was no database named 'example' on the server before I ran this example.

[Saghm Rossi]

Thanks for the self-contained example here! This makes it much clearer for me.

This seems to be the same issue as #244; if the call to insert_one is changed to the following, then it shows the error in the write_exception field:

dbg!(users.insert_one(
    doc! {
        "username": username,
    },
    None,
).unwrap()

Output:

[src/main.rs:9] users.insert_one(doc!{"username" : username , }, None) = Ok(
    InsertOneResult {
        acknowledged: true,
        inserted_id: None,
        write_exception: Some(
            WriteException {
                write_concern_error: None,
                write_error: Some(
                    WriteError {
                        code: 11000,
                        message: "E11000 duplicate key error collection: example.users index: username_1 dup key: { : \"example-user\" }"
                    }
                ),
                message: "WriteError { code: 11000, message: \"E11000 duplicate key error collection: example.users index: username_1 dup key: { : \\\"example-user\\\" }\" }"
            }
        )
    }
)

[Andrew J Westlake]

Awesome, the write_exception was exactly what I needed. I should have dug into the docs a bit more. Thanks for your help!

Should we leave this issue open or close it in favor of #244?

[Saghm Rossi]

No worries! At the very least, I think that we should document this method (and any others where this can occur) that write errors will not cause an Err variant to be returned.

[Saghm Rossi]

Hello everyone! We've just released an alpha of an official MongoDB Rust driver; you can read about it here! Because of this, we won't be updating this driver anymore, and the repository will be made read-only as soon as I get finished posting these messages. Thanks for using this driver, and we hope you try out the new one!

[Saghm Rossi]

Apologies, the blog post link is this: https://www.mongodb.com/blog/post/announcing-the-official-mongodb-rust-driver