name: chatwoot-whatsapp-masked-phone-debug

description: Debug Chatwoot WhatsApp webhook when messages arrive but don't create conversations — root cause is masked phone number in database vs real number from Meta API.

Chatwoot WhatsApp — Debugging "Messages Not Arriving" (Masked Phone Number)

Symptom

- WhatsApp webhook configured correctly in Meta Developers
- Webhook URL verified successfully (hub.challenge returns correctly)
- But incoming WhatsApp messages don't create conversations in Chatwoot
- No errors in Chatwoot UI, but messages never appear

Root Cause

The actual DB storage format: Chatwoot stores phone_number in the DB without the + prefix and without masking (e.g., 5511950801701). The masked display +551**1701 is only a UI helper — not what's queried at runtime.

However, the real problem is worse than a display mismatch: the phone number stored in Chatwoot's DB may not match the display_phone_number in the Meta API payload.

Example of the actual mismatch discovered:

- DB stored phone: 5511950801701 (Chatwoot channel setup)
- Meta API display_phone_number in webhook: 5511915081701
- These are different numbers — digits 4-6 differ: 950 vs 915

When Chatwoot does phone_number.to_s.gsub('+', '') on the incoming payload, it produces 5511915081701, which doesn't match the DB value 5511950801701. The channel lookup returns nil and the message silently lands in the Sidekiq dead queue.

Why the numbers differ: The Meta WhatsApp Business Account may have had its phone number updated in Meta's system since the Chatwoot channel was first configured, or the wrong phone was registered. The phone_number_id (e.g., 812251211981254) is the stable, authoritative identifier that always matches.

Silent failure pattern: whatsapp_events_job.rb catches all exceptions broadly, so failures in get_channel_from_wb_payload return nil with no error raised — messages disappear into the dead queue with no visible error in logs.

New Diagnostic Pattern Discovered (2026-05-27)

POST works but GET verify fails (401 "wrong verify token")

If:

- curl -X POST https://chat.rochasalesseguros.com.br/webhooks/whatsapp/+5511955701701 → 200 (messages created in DB)
- curl -k 'https://chat.rochasalesseguros.com.br/webhooks/whatsapp/+5511955701701?hub.verify_token=TOKEN&hub.mode=subscribe&hub.challenge=test123' → 401 "wrong verify token"
- But the token in the DB matches the token you're sending

Cause: The Chatwoot valid_token? reads from channel.provider_config['webhook_verify_token'] at runtime. The 401 means the channel lookup is working for POST but failing for GET. Since the get route passes params[:phone_number] directly and the channel IS found for POST, the token is definitely wrong in the actual request being made by Meta.

Key insight: The Meta Developers portal's webhook setup screen ALLOWS you to set any arbitrary verify token — it does NOT validate that the token matches what's in Chatwoot's DB. If you (or someone) previously typed a wrong token in the Meta portal, the validation will always fail even if Chatwoot has the correct one. The verify token in Meta portal and Chatwoot DB MUST match.

Fix: The verify token in Meta Developers portal (App → Webhooks → WhatsApp → "Verify token" field) must be set to the EXACT value from Chatwoot's DB:

`sql

SELECT provider_config->>'webhook_verify_token' FROM channel_whatsapp WHERE phone_number LIKE '%1701%';

POST working but no conversations created

If POST returns 200 but no conversation is created: Messages may be landing in the Sidekiq dead queue. Check:

`sql

docker exec root-rails-1 bundle exec rails runner "puts Sidekiq::DeadSet.new.size"

docker exec root-rails-1 bundle exec rails runner "Sidekiq::DeadSet.new.each_with_index.each { |j, i| puts \"#{i}: #{j.args}\" }"

Two Meta Business Apps registered for the same phone number

If you have multiple Meta apps with different webhook URLs (you confirmed: App 1 → n8n, App 2 → ComercialRS), the Meta Business Account uses ONE primary webhook URL per phone number. Check which App is configured as the "receive messages" destination in https://business.facebook.com → Messaging → Receive messages.

Two-Part Fix

Fix 1: `app/controllers/webhooks/whatsapp_controller.rb`

In valid_token?, after find_by(phone_number: phone_number) fails, fall back to phone_number_id:

`ruby

def valid_token?(params)

phone_number = params[:phone_number]

token = params[:hub_verify_token]

channel = Channel::Whatsapp.find_by(phone_number: phone_number)

# Fallback: find by phone_number_id in provider_config when phone_number is masked

if channel.nil? && params[:entry].present?

phone_number_id = params[:entry].first.dig(:changes, 0, :value, :metadata, :phone_number_id)

channel = Channel::Whatsapp.all.find do |c|

c.provider_config['phone_number_id'] == phone_number_id

end

return false unless channel

ActiveModel::SecurePassword::BCrypt.new.valid?(channel.provider_config['webhook_verify_token'], token)

end

Also in process_payload, add the same fallback before processing messages:

`ruby

channel = Channel::Whatsapp.find_by(phone_number: phone_number)

if channel.nil?

phone_number_id = payload.dig(:entry, 0, :changes, 0, :value, :metadata, :phone_number_id)

channel = Channel::Whatsapp.all.find do |c|

c.provider_config['phone_number_id'] == phone_number_id

end

Fix 2: `app/jobs/webhooks/whatsapp_events_job.rb`

In find_channel_from_whatsapp_business_payload, after find_by(phone_number: phone_number) returns nil:

`ruby

channel = Channel::Whatsapp.find_by(phone_number: phone_number)

if channel.nil?

phone_number_id = message[:metadata][:phone_number_id]

channel = Channel::Whatsapp.all.find do |c|

c.provider_config['phone_number_id'] == phone_number_id

end

Fix 3: nginx WebSocket support

For ActionCable (required for live chat UI):

`nginx

location /cable {

proxy_pass http://unix:///var/run/chatwoot.socket;

proxy_http_version 1.1;

proxy_set_header Upgrade $http_upgrade;

proxy_set_header Connection "upgrade";

proxy_set_header Host $host;

}

Meta Webhook Registration (422 Error)

If registering the webhook in Chatwoot Settings → Inbox → WhatsApp gives 422:

1. The Meta app must be subscribed to receive messages in Meta Developers Portal

2. Go to: https://developers.facebook.com/apps/YOUR_APP_ID/webhooks/

3. Under "Subscription Fields", check: messages, message_deliveries, message_reads

4. Then try the "Subscribe" button in Chatwoot

Meta API — Discover the Real Phone Number

When phone numbers in Chatwoot and Meta don't match, query Meta directly:

`bash

Get the phone_number_id from Chatwoot DB first, then query Meta

curl -s "https://graph.facebook.com/v18.0/PHONE_NUMBER_ID?access_token=ACCESS_TOKEN" | jq

This returns the display_phone_number that Meta has on file. Compare it with what's in Chatwoot:

`sql

SELECT id, phone_number, provider_config FROM channel_whatsapps WHERE id = INBOX_ID;

If they differ, the Meta system has a different number than what was configured in Chatwoot. Use phone_number_id as the stable fallback key.

Dead Queue — Inspect and Retry

Messages that fail channel lookup land in Sidekiq's dead set and are lost after 24 hours. To inspect:

`bash

Inspect dead queue size and jobs

docker exec root-rails-1 bundle exec rails runner "load '/tmp/check_dead.rb'"

Where check_dead.rb contains:

puts Sidekiq::DeadSet.new.size

Sidekiq::DeadSet.new.each_with_index.each { |j, i| puts \"#{i}: queue=#{j.queue} jid=#{j.jid} args=#{j.args rescue 'N/A'}\" }

Retry ALL dead jobs (low priority queue)

docker exec root-rails-1 bundle exec rails runner "load '/tmp/retry_all_dead.rb'"

Retry a specific job by JID

docker exec root-rails-1 bundle exec rails runner "Sidekiq::DeadSet.new.find { |j| j.jid == 'JID_HERE' }&.retry"

Deploying Code Fixes to Docker Containers

If Chatwoot is running as a Docker image (not in a git repo), use docker cp to inject fixed files:

Step 1 — Write the fix locally

Use write_file to create the fixed Ruby file on your local machine.

Step 2 — Upload via scp

`bash

sshpass -p 'PASSWORD' scp -o StrictHostKeyChecking=no /tmp/whatsapp_events_job.rb root@VPS_IP:/tmp/

Step 3 — Copy into container

`bash

sshpass -p 'PASSWORD' ssh root@VPS_IP "docker cp /tmp/whatsapp_events_job.rb CONTAINER_NAME:/app/app/jobs/webhooks/whatsapp_events_job.rb"

Step 4 — Restart container

`bash

sshpass -p 'PASSWORD' ssh root@VPS_IP "docker restart CONTAINER_NAME"

Note: rails runner quoting is fragile with nested quotes — use load "/tmp/script.rb" pattern instead of inline Ruby.

Server Info

- VPS: 31.97.243.106, root password: Marcia19671951@
- Chatwoot Docker containers: root-rails-1, root-sidekiq-1
- Domain: chat.rochasalesseguros.com.br
- PostgreSQL: container root-postgres-1, database chatwoot_production
- Redis: redis://redis:6379

Diagnostic Commands

`bash

Check webhook is receiving messages

docker logs --since 5m root-rails-1 2>&1 | grep 'webhooks/whatsapp'

Check Sidekiq processing

docker logs --since 5m root-sidekiq-1 2>&1 | grep -i 'whatsapp'

Find the real phone number from Meta API

curl -s "https://graph.facebook.com/v18.0/PHONE_NUMBER_ID?access_token=TOKEN" | jq

Check dead jobs (messages that failed to process)

docker exec root-rails-1 bundle exec rails runner "puts Sidekiq::DeadSet.new.size"

Retry dead jobs

docker exec root-rails-1 bundle exec rails runner "Sidekiq::DeadSet.new.retry_all"

Related Skills

- chatwoot-nginx-underscore-header-fix — WebSocket 401 errors from nginx dropping underscore headers
- chatwoot-whatsapp-webhook-debug — General WhatsApp webhook debugging on VPS

Chatwoot WhatsApp — Debugging "Messages Not Arriving" (Masked Phone Number)

Symptom

Root Cause

New Diagnostic Pattern Discovered (2026-05-27)

POST works but GET verify fails (401 "wrong verify token")

POST working but no conversations created

Two Meta Business Apps registered for the same phone number

Two-Part Fix

Fix 1: app/controllers/webhooks/whatsapp_controller.rb

Fix 2: app/jobs/webhooks/whatsapp_events_job.rb

Fix 3: nginx WebSocket support

Meta Webhook Registration (422 Error)

Meta API — Discover the Real Phone Number

Get the phone_number_id from Chatwoot DB first, then query Meta

Dead Queue — Inspect and Retry

Inspect dead queue size and jobs

Where check_dead.rb contains:

puts Sidekiq::DeadSet.new.size

Sidekiq::DeadSet.new.each_with_index.each { |j, i| puts \"#{i}: queue=#{j.queue} jid=#{j.jid} args=#{j.args rescue 'N/A'}\" }

Retry ALL dead jobs (low priority queue)

Retry a specific job by JID

Deploying Code Fixes to Docker Containers

Server Info

Diagnostic Commands

Check webhook is receiving messages

Check Sidekiq processing

Find the real phone number from Meta API

Check dead jobs (messages that failed to process)

Retry dead jobs

Related Skills

Fix 1: `app/controllers/webhooks/whatsapp_controller.rb`

Fix 2: `app/jobs/webhooks/whatsapp_events_job.rb`