Ansible lookup plugin is back, rewritten for Passbolt 5

Introduction

Back in 2022, Jean-Christophe published an article about managing secrets in Ansible using passbolt, along with a community lookup plugin. A lot has happened since then, including some pretty big breaking changes: metadata is now encrypted, new resource types are available (with custom fields, multiple URIs, etc.) and the availability of a new account portability file called the account kit, currently used by the windows app. As the old plugin couldn't handle any of this, we took on the mission to give it much needed facelift.

This article walks you through what changed, as well as what the plugin does under the hood. It comes with a ready-to-use docker playground so you can try it out in minutes!

What changed (and what didn't)

Considering the extensive changes, we decided to start a new plugin rather than iterate on the existing one. This new plugin is therefore not a drop-in replacement for the old one. Some things got better, some features have been removed or aren't there yet. Here’s the honest breakdown.

What's new: 

• Configuration via the account kit file format: the account kit simplifies what was previously a manual config and passes your OpenPGP private key and passphrase as environment variables. To use the account kit, you will need to download it from the web UI in Manage account > Desktop app setup > Download your account kit. It bundles everything needed for authentication in a single signed file. 
• Sensitive account data storage: the account kit and passphrase can be stored as ansible-vault variables instead of environment variables, to better protect this sensitive data.
• Encrypted metadata support: The plugin handles both shared and personal metadata-encrypted resources, which is the new default in passbolt version 5. If you don't know what metadata encryption is yet, don't worry, we'll get there.
• Support for new resource types: The plugin returns the full resource as a single dictionary item: password, username, URI(s), TOTP, secure notes, custom fields, all in one object. You pick what you need in your playbook.

What's not there:

• Search: The old plugin lets you search by resource name, username, or URI. Handy, but also ambiguous when you have three resources called "db-production". The new one requires the resource UUID. More explicit, less guessing, but it means you need to grab those UUIDs upfront. Using the user interface, you can click on a resource and the web browser URL will share the UUID as {{your_domain}}/app/passwords/view/UUID
• Field selection: Instead of returning just a password or a description, e.g. a field of a given record, the plugin gives you the entire decrypted resource and lets your playbook decide what to use. You can definitely think of it as one API call that returns the entire record, and your playbook is the filter.
• Resource creation: the community plugin supported creating and updating resources through Ansible modules. This plugin is read-only for now. It's a lookup plugin focused on one thing: securely retrieving secrets. We took this direction for a clearer separation of concern: a lookup, as the name suggests, is to locate and should maybe not be expected to create or update records.
• Galaxy Publishing: Right now you install it from the git repository. Galaxy publishing is on our radar, but we're not there today.

Under the hood

So what actually happens when you call:

lookup('passbolt.passbolt_lookup.passbolt_lookup', '<uuid>')

Quite a lot actually! If you're the kind of person who likes to know what's going on behind the curtain, this section is for you. If not, feel free to skip straight to the docker playground. And if you are really into it, the full public technical specifications are available here for your perusal.

The account kit, your new identity file

The account kit is a base64-encoded, OpenPGP signed message that is itself composed of a JSON message. This JSON message contains your private key, the server's public key, the domain, your identity, and a security token. All of that in one file, signed with your key so integrity can be validated.

The first thing the plugin does is verify the PGP signature of the kit. If it's been tampered with, the plugin stops right there, no network call is made. It then imports your private key, validates your passphrase and verifies the signature a second time against your specific key fingerprint. Only after that does it import the server's public key and call the server to confirm the fingerprint matches. Think of it as a bouncer checking your ID, your invitation, and calling the host to confirm you're actually on the list.

The grown-up crypto choice

One thing you might notice if you dig into the code: all crypto operations goes through GnuPGService, which is a thin wrapper around the Gnupg binary via python-gnupg. 

The old plugin used PGPy, a pure Python implementation of OpenPGP, but to our knowledge it is not actively maintained anymore. GnuPG is battle-tested, actively maintained, and it's the same tool passbolt itself relies on.

The service handles key imports, encryption, decryption, signing, and verification. Every operation checks the result and throws a dedicated exception with a human-readable message if something goes sideways. No silent failures.

JWT authentication

If you've read my previous article about JWT authentication in passbolt, this will sound familiar. The old plugin used GPGAuth, the new one uses GpgJwtAuth.

In short, the plugin creates a challenge and then encrypts it with the server's public key, signs it with yours, and sends it to POST /auth/jwt/login.json. The server checks if it's legit by decrypting and verifying the signature of your challenge. The server sends back its own encrypted response with your challenge, an access token and a refresh token, encrypted with your public key and signed with the server key. The Ansible plugin decrypts that, checks the signature, the challenge, the tokens, checks everything matches and boom, you're authenticated.

From that point on, every API call carries a Bearer token in the HTTP header. When the plugin is done, it calls POST /auth/jwt/logout.json to kill the session server-side. 

Fetching and decrypting a resource and its secret

Now the fun part. This is where v5 is fundamentally different from older versions. In the old days, resource metadata (name, username, URIs) was stored in plaintext on the server. That means that only the secret data such as the password was encrypted. In v5, metadata can be encrypted too. So when your playbook asks for a resource, the plugin has to peel two layers.

The first one, it fetches the resource from GET /resources/{uuid}.json?contain[secret]=1. This gives back the encrypted metadata, the encrypted secret, and a metadata_key_type field that tells the plugin how the metadata was encrypted. If metadata_key_type is user_key, it means the metadata was encrypted with your personal key. Easy, the plugin just decrypts and verifies it with your key.

If it's shared_key, things get more interesting. The metadata was encrypted with a shared key that multiple users can access. The plugin fetches that shared key from GET /metadata/keys.json?contain[metadata_private_keys], finds the copy that's encrypted for your user, verifies it with your key, decrypts it, imports it into the GnuPG keyring, and then decrypts the metadata. Those shared keys are cached in memory for the session, so this dance only happens once per key even if you're fetching multiple resources.

For personal resources, the plugin decrypts and verifies the secret signature with your key. For shared resources, it fetches the public key of the user who last modified the secret and verifies the signature against their key ensuring the secret was indeed signed by the person who encrypted it.

Then, finally, the plugin merges everything into a single flat dictionary. Metadata such as name, username, description, URIs, custom fields keys and icon come from the metadata object. On the other end, password, note, TOTP, and custom field values come from the secret object. 

Regarding custom fields specifically, the result only includes fields that actually have content, so you won't see a bunch of null values cluttering your playbook output. Then strips any null or empty fields, so your playbook only sees what's actually there.

Try it yourself

Reading about it is cool, but running it is better. I created a demo repository with a full docker compose stack so you can see all of this in action without installing anything on your machine.

git clone [email protected]:qntoni/ansible-lookup-plugin-demo.git
cd lab-passbolt-ansible-lookup; docker compose build; docker compose up -d
docker compose run ansible bash -c "ansible-playbook playbooks/playbook.yml"

That's it. Three commands and you should see something like this:

TASK [Display simple password]
ok: [localhost] => {
    "msg": [
        "Name:     Tatooine Moisture Farm",
        "Username: owen.lars",
        "Password: M01stur3_V4p0r!8"
    ]
}
TASK [Display password with TOTP]
ok: [localhost] => {
    "msg": [
        "Name:        Imperial Academy Portal",
        "Username:    cadet.tk421",
        "Password:    Tr00p3r!Dut7$",
        "TOTP secret: GEZDGNBVGY3TQOJQ",
        ...
    ]
}

The stack includes a pre-populated Passbolt CE instance with sample resources covering every type (simple passwords, passwords with notes, TOTP, custom fields, multiple URIs), and an Ansible container with the plugin pre-installed and a ready-to-go example playbook. The account kit and passphrase are already encrypted with ansible-vault in the repository, with the vault password committed alongside it (because it's a demo, please don't do this in production).

If you want to go beyond the playbook and actually browse the resources in the user interface, add passbolt to your /etc/hosts:

127.0.0.1      passbolt

Then head to https://passbolt in your browser. The demo comes with a whole Star Wars cast of users (Anakin, Yoda, Vader, Obiwan, Luke, Leia...) and their OpenPGP keys are in the /gpg folder of the repository. To recover an account e.g., anakin, run:

docker compose exec -ti passbolt su -s /bin/bash -c "./bin/cake passbolt recover_user -c -u [email protected]" www-data

This will give you a recovery link. The passphrase for each user is simply their email. Import the matching private key from the gpg/ folder and you're in. 

SDK enthusiasts: A new hope

If you’ve been hanging around the community forum, you've probably seen the requests “But, when will there be an official SDK? Any plans for a proper client library?” Well, we haven’t been ignoring those threads.Here is the thing, one of the reasons this plugin took the time it took to implement is that we didn’t just slap together a script that calls the API. If you look at the codebase or even the technical specifications. The internals are organized into clear modules such as auth, account, cryptography, entities, http and the passbolt_api_client.py. Today, they live inside the Ansible collection but the way they’re structured, with clean boundaries and no Ansible-specific dependencies in the core logic, that’s intentional. 

We designed the internals with a bigger picture in mind. The authentication flow, the metadata decryption, the resource fetching, all of that is generic enough to live outside of the lookup. Think of the current plugin as the first consumer of building blocks that could eventually serve a broader SDK.

Right now, we’re not going to drop an ETA or promise anything specific here, but what I can say is that the foundations are there, and this plugin is the proof that those foundations work. If you are someone who has been waiting for a proper way to talk to the passbolt API programmatically in python without reinventing the wheel every time, you know now it’s very much on our radar. Stay tuned! Or even as a wise green man once said: “Patience you must have, my young padawan”. 

Conclusion

The new passbolt’s official Ansible lookup plugin is a fresh start built for v5. It doesn't do everything the old ecosystem did. No write operations, no search by name, and it is not on Galaxy yet. 

But it handles the core use case well: securely retrieving secrets from passbolt in your playbooks, with full support for the new encrypted metadata format, TOTP, custom fields, and multiple URIs.

Give the playground a spin, break things, and if you have feedback or ideas drop by the community forum. We'd love to hear what you think and what you'd want to see next.

Oh and one last thing: the plugin is now part of our UATs. Which means it gets tested against every new release before it ships. So if a future version introduces breaking changes (Passbolt 6? 👀 who knows), the plugin will be updated accordingly. No more "this is a community plugin" excuse for us! 

Thanks for your patience on this one. We know it took a while to get here, and we hope it was worth the wait. We always try to find the right balance between security and usability, but with a plugin that handles your secrets, we’ll do our best so that security always comes first!