Set up content-safety scanning (Google Cloud)

This guide turns on Bommel's automatic safety checkers. These are four Google services that read a bookmark before it can appear in public and quietly raise a hand if it looks like something we never allow — sexual content, content involving minors, violence, or dangerous/malware links.

Set up properly, they don't just remove bad content: they let clean submissions publish themselves, so you aren't approving thousands of bookmarks by hand. That last part is switched on separately, at the very end, once you've seen the checkers agree with your own judgement.

You do not need to be a programmer. If you can copy and paste and click blue buttons, you can do this. We'll go one click at a time.

You only do this once. After it's set up, it runs by itself forever.

The big picture (read this first)

Bommel already talks to Google for your sign-in and cloud storage (that's "Firebase"). Firebase lives inside a bigger house called Google Cloud. We're just switching on three extra helpers in that same house:

• Natural Language API — reads words (titles, notes, tags) and flags hateful or sexual text.
• Vision API — looks at pictures (cover images) and flags adult or violent images (this is the "SafeSearch" check).
• Web Risk API — checks a link's address against Google's list of known dangerous sites (malware, phishing, scams).
• Gemini — actually opens the page the bookmark points at and judges what's on it.

Because Firebase and Google Cloud are the same project, Bommel already has an ID badge (called a "service account") that can use the first three. So the whole job is really just: turn the helpers on, make sure the badge is allowed to use them, paste in a Gemini key, and flip one switch in Bommel.

Why the fourth one matters most

The first three only ever see what the person saving the bookmark typed, plus the link's address. None of them opens the page. So a bookmark pointing at a pornographic site, titled "Interesting article", with no cover image, looks completely clean to all three — Web Risk only knows about malware and scams, not adult content.

Gemini closes that hole by reading the destination page itself. It is the reason automatic approval can be trusted at all, which is why Bommel refuses to auto-approve anything when it hasn't run or when it reports that it couldn't read the page.

What you'll need

• The Google account you used to create your Firebase project.
• About 15 minutes.
• A credit or debit card. Google requires one to switch these on, but there's a big free monthly allowance and you can set a hard spending cap so you're never surprised (we cover that at the end).

Step 1 — Open Google Cloud and pick the right project

1. Go to console.cloud.google.com and sign in with the same Google account you use for Firebase.
2. At the very top of the page, next to the words Google Cloud, there's a project picker (a dropdown that shows a project name). Click it.
3. In the little window that opens, find and click the project that matches your Bommel/Firebase project. The name usually matches what you see in the Firebase console.

How do I know it's the right one? Open your Firebase console in another tab, click your project, then the gear icon → Project settings. The Project ID shown there must match the one you pick in Google Cloud. They have to be the same project.

You'll know you're in the right place when that project name stays shown at the top.

Step 2 — Turn on billing (with a safety cap)

These three APIs need "billing enabled." That sounds scary, but it just means Google needs a card on file. You'll stay inside the free allowance for a long time, and we'll add a cap so it can never run wild.

1. In the search bar at the top, type Billing and click Billing.
2. If it says This project has no billing account, click Link a billing account → Create billing account, and follow the prompts to add your card.
3. Once billing is linked, set a safety net: search for Budgets & alerts (under Billing) → Create budget. Set a small monthly amount (for example, $5) and tick the boxes to email you at 50%, 90%, and 100%. This warns you long before any real charge.

You will almost certainly pay nothing at small scale — see Costs near the bottom. The budget is just peace of mind.

Step 3 — Turn on the Natural Language API (reads text)

1. In the top search bar, type Cloud Natural Language API and click it.
2. You'll land on a page with a big blue Enable button. Click Enable.
3. Wait a few seconds. When it says the API is enabled, you're done with this one.

That's the whole step. No keys to copy.

Step 4 — Turn on the Vision API (looks at pictures)

1. In the top search bar, type Cloud Vision API and click it.
2. Click the blue Enable button.
3. Wait for it to say enabled.

Step 5 — Turn on the Web Risk API (checks links)

1. In the top search bar, type Web Risk API and click it.
2. Click the blue Enable button.
3. Wait for it to say enabled.

You've now switched on all three helpers. 🎉

Step 6 — Permissions (usually nothing to do)

In most projects you can skip this step. Try Step 9 first, and only come back if a checker reports "No permission".

Here's why there's so little to do, because it's genuinely surprising: none of these three APIs has its own IAM role. If you go looking for "Web Risk User" or "Cloud Vision API User" in the role picker, you won't find them — they don't exist. Google says so explicitly for Web Risk: "Web Risk does not require any special permissions to call any APIs. This means you can use service accounts with no IAM roles." Vision and Natural Language work the same way: they authorize on the OAuth scope of the access token, not on a role, and the Firebase Admin SDK already requests the cloud-platform scope that all three accept.

So for these three, enabling the API is the permission.

The one role that can be missing is Service Usage Consumer, which governs whether an account may call any enabled API in the project. If a checker shows "No permission":

1. In the top search bar, type IAM and click IAM & Admin → IAM.
2. Find the member ending in `iam.gserviceaccount.com` that matches the email in your FIREBASE_ADMIN_CLIENT_EMAIL setting.
3. If its Role column already says Editor or Owner, permissions aren't your problem — re-check Steps 3–5 instead.
4. Otherwise click the pencil (Edit) icon → Add another role → Service Usage Consumer → Save.

Why this is the right role (least privilege): it grants permission to use APIs that are already switched on, and nothing else — no ability to enable new APIs, read data, or change the project. Adding Editor would work too, but it hands over far more than these checks need.

Step 7 — Get a Gemini key (the checker that reads the page)

This one doesn't use the service-account badge, so it needs its own key.

1. Go to [aistudio.google.com/apikey](https://aistudio.google.com/apikey) and sign in with the same Google account.
2. Click Create API key and pick your existing Firebase/Google Cloud project so the usage lands on the billing account you already capped in Step 2.
3. Copy the key. You'll paste it in the next step.

If you already set GEMINI_API_KEY for AI cover images, that same key works here — you don't need a second one.

Keep this key secret. It goes in your server settings only. It is never sent to the browser and never appears in a page's source.

Step 8 — Flip the switch inside Bommel

The helpers are on, but Bommel keeps safety scanning off by default until you deliberately turn it on (this is a "fail-safe": if something isn't set up, it stays cautious rather than assuming it's fine).

Add these lines to your Bommel server settings (your .env.local for local testing, or your hosting provider's Environment Variables screen for the live site):

MODERATION_SCANNING_ENABLED=true
GEMINI_API_KEY=your-key-from-step-7

Then restart the app (or redeploy, if it's live) so it picks up the new setting.

The service account (only if you don't have one yet)

The other three checkers use Bommel's "ID badge" — a Firebase service account. If FIREBASE_ADMIN_CLIENT_EMAIL and FIREBASE_ADMIN_PRIVATE_KEY are already filled in, skip this; they're shared with other server features and won't need changing.

To create one:

1. Go to the [Firebase console](https://console.firebase.google.com) and open your project.
2. Click the gear icon (top left, next to Project Overview) → Project settings.
3. Open the Service accounts tab.
4. Click Generate new private key → Generate key. A .json file downloads.
5. Open that file in a text editor. You need three values from it:

The client_email looks like firebase-adminsdk-a1b2c@your-project.iam.gserviceaccount.com. That's also the member you'd look for in Step 6.

The private key is the fiddly one. In the JSON it's a single long line beginning -----BEGIN PRIVATE KEY-----\n with literal \n sequences in it. Copy the value exactly as it appears, including the \n characters and the BEGIN/END lines — don't convert them into real line breaks. In .env.local, wrap it in double quotes:

FIREBASE_ADMIN_PRIVATE_KEY="-----BEGIN PRIVATE KEY-----\nMIIEv...\n-----END PRIVATE KEY-----\n"

On Vercel, paste the same value into the variable's value box (quotes optional there).

Skip the copying (easier on your own machine)

Rather than transcribing three values, you can point Bommel straight at the downloaded file — same credential, none of the \n hazard:

GOOGLE_APPLICATION_CREDENTIALS=C:\keys\bommel-service-account.json

Bommel uses this only when the three variables above aren't all set, and the file has to exist — there's no silent fallback to some other credential. Keep the .json outside your project folder so it can't be committed by accident. For hosted deployments there's no file to point at, so use the three variables there.

Treat this file like a password. It grants full admin access to your Firebase project. Never commit it, never put it in a NEXT_PUBLIC_ variable, and delete the downloaded .json once you've copied the values out. If it ever leaks, return to the Service accounts tab and generate a new key — that's also where you'd revoke the old one.

On Vercel: Project → Settings → Environment Variables → Add, set the name to MODERATION_SCANNING_ENABLED and the value to true, save, then Redeploy.

Notice what you have not turned on yet: automatic approval. Scanning is now removing the worst content and pre-tagging the rest, but a human still approves everything. That's intentional — see Step 10.

Step 9 — Check that it's working

This happens inside Bommel — not in Google Cloud or Firebase. In your own site, sign in as an admin and go to:

• `/admin/moderation` (for example https://your-site.com/admin/moderation), or
• Settings → Moderation → Open moderation dashboard, which is the same page.

Scroll to the Safety checkers panel. It calls all four services right then and tells you the state of each:

• Live — working. ✅
• Not enabled — the API isn't switched on for this project. Go back to Steps 3–5.
• No permission — the API is on, but Bommel's badge can't call it. Add Service Usage Consumer (Step 6). This one is rare.
• Not configured — a missing environment variable (the service account, or the Gemini key).
• Out of quota — check billing.

Each row tells you the exact next step. Fix something, hit Re-check, and you'll see it flip to Live within a minute or two — permission changes sometimes take a moment.

That panel is the fastest way to confirm the setup. If you'd also like to see it end to end, publish a harmless bookmark from a test account and watch it arrive in the dashboard with a verdict attached.

Step 10 — Let clean bookmarks publish themselves

This is the step that removes the daily approval work — and the one worth being careful about, because it's the only setting that lets content reach the public without you.

First, run it in shadow mode for a week. You've already done this: scanning is on, auto-approval is off. Every submission now carries a full verdict, so as you work through the dashboard you can see what the pipeline would have done and compare it to what you actually decide. If it agrees with you consistently, it's ready.

When you're satisfied, add:

MODERATION_AUTO_APPROVE=true

From then on, a submission that comes back clean on every check that matters publishes immediately. Everything else still comes to you.

"Every check that matters" is enforced rather than assumed:

• Gemini must always have run, and must confirm it really read the destination page. This one is never waived — it's the only checker that sees the page itself.
• Web Risk and Vision must have run if they're reachable. A checker that's set up but slow, erroring, or out of quota counts as unchecked, not as fine, so the bookmark queues for you rather than slipping out.
• A checker you never enabled is skipped rather than treated as a failure. Otherwise a setup without, say, Web Risk would approve nothing at all, forever, with no visible error.

That last point has a consequence worth being deliberate about. If Vision isn't enabled, nothing looks at cover images — a bookmark pointing at a perfectly clean page could still carry an explicit picture, and approval won't wait for a check that doesn't exist. The Safety checkers panel warns you when you're in that state. For a public directory, enable Vision.

The good news is it's self-correcting: turn Vision or Web Risk on and Bommel starts requiring them automatically, within a minute, with no redeploy.

If you turn auto-approval on and nothing ever gets approved, open the Safety checkers panel — it will name the checker that's holding things up.

To stop it at any time, set it back to false and redeploy. New submissions go back to waiting for you; anything already approved stays up.

There's also MODERATION_CLASSIFIER_MODEL if you ever want a different Gemini model. Leave it empty unless you have a reason — the replacement must support both structured output and URL Context, and if it doesn't, the classifier fails closed and nothing will auto-approve.

Costs — the honest version

All four have a monthly free allowance, and at typical small-community sizes you'll likely stay inside it:

• Natural Language and Vision: the first 1,000 checks per month are free, then a few dollars per additional thousand.
• Web Risk: has a generous free monthly quota for lookups.
• Gemini: the classifier uses the cheapest current small model with a very short reply. Even at tens of thousands of bookmarks a month this is cents, not dollars — reading the linked pages is the main cost, and it's still small.

The budget alert from Step 2 emails you well before any real spend. If you ever want to pause everything, set MODERATION_SCANNING_ENABLED=false and redeploy — scanning stops immediately, and because approval requires a completed scan, auto-approval stops with it.

Prices change over time. For exact, current numbers, search "Google Cloud [API name] pricing" — but for a growing community, expect $0 to a few dollars a month.

If something goes wrong

Start with the Safety checkers panel (Step 9) — it names the problem and the fix for each service. Then:

• "Not enabled" → re-check Steps 3–5. Make sure you enabled it in the same project (Step 1).
• "No permission" → Step 6; grant Service Usage Consumer. Don't go looking for a "Web Risk User" or "Cloud Vision API User" role — those don't exist. New permissions can take a minute or two.
• "Not configured" → the server is missing FIREBASE_ADMIN_CLIENT_EMAIL / FIREBASE_ADMIN_PRIVATE_KEY (for the three Google APIs) or GEMINI_API_KEY. See Step 8 for where those come from. Add them and redeploy.
• "Billing account required" → finish Step 2; an API won't run without billing linked, even inside the free tier.
• Wrong project → the number-one mistake. Double-check the Project ID matches between Firebase and Google Cloud (Step 1).
• Auto-approval is on but nothing ever gets approved → the fail-safe working, not a bug: a required checker isn't answering, so Bommel refuses to publish content it can't vouch for. The panel flags exactly which one.
• The moderation dashboard shows "Sign in to continue" even though you're signed in → your server session cookie didn't get established. The page tries to repair it automatically; if it can't, sign out and back in. On a local machine it usually means the Firebase Admin variables are empty.

A note on safety and privacy

• The API checks look at content someone is trying to make public. Separately, a fast word-and-domain check runs on every save, including private ones, because Bommel isn't a place to keep a private collection of prohibited material either. It runs on the device and is enforced again by the database, so a modified app can't store this material on our servers. That check needs no setup and costs nothing.
• The most serious category — anything involving minors — is treated with zero tolerance: the content is refused every time, the first detection issues a single final warning, a second permanently closes the account, and an incident record is written so the operator can file the report US law requires (see Reporting content & the moderation queue).
• Bommel never logs the actual text or images it checks; it only records the outcome (short reason codes, and one neutral sentence describing the page) so your users' content stays private.

That's it — you've given Bommel a tireless, around-the-clock safety net. Next, see Reporting content & the moderation queue to learn how you (and your community) review anything the checkers flag.