name: google-drive-shared-folder-migration

description: Migrate documents from Google Drive shared folders ("Proprietário oculto") to DocCorretor using Playwright browser automation

tags: [google-drive, migration, playwright, doccorretor]

Google Drive Shared Folder Migration to DocCorretor

Context

Migrate documents from Google Drive shared folders ("documentos clientes") to DocCorretor (Supabase Cloud + VPS storage). The Drive is organized as documentos clientes/{year}/{month}/{client folders}/.

Key Findings

Google Drive HTML Structure

• - Folder IDs: Stored in data-id attribute on TR elements, NOT on DIVs or parent A elements.
• - Folder names: Found in div[aria-label] inside the TR, with format {client_name} Pasta compartilhada.
• - Navigation: Double-click on div[aria-label="MM/YYYY ..."] to navigate into month folders.
• - Year folder IDs (from root 1zVhApvpFM937qxsvsz37WCcUD-7f2zUP):

- 2023: 1nObuuq8AhcUo-O96UdhaTBbrjaRzI4lP

- 2024: 1CcabJC1HF1V-tVUlq9IjkpCLNYRMN_l5

- 2025: 17-KPk4ttAvtmz2hCXarIiW4O0MPh4rGZ

- 2026: 1rlhX77Rq5F40qctMFMP7k2AxD5-U3rSS

Critical: gdown Fails on "Proprietário Oculto" Folders

• - gdown --folder only works on publicly accessible folders.
• - Folders shared as "Proprietário oculto" (hidden owner) return: Failed to retrieve file url: Cannot retrieve the public link of the file.
• - Solution: Must use Playwright browser automation with authenticated session to scrape/download files.
• - Browser user data dir: /tmp/gdrive_profile (Xvfb for headless: xvfb-run -a python3).

Extracting Client Folders via JavaScript (Playwright)

`javascript

// Use page.evaluate() in Playwright — DIV aria-label approach fails for TR elements

clients = page.evaluate('''() => {

const results = [];

const seen = new Set();

const trs = document.querySelectorAll("tr[data-id]");

trs.forEach(tr => {

const id = tr.getAttribute("data-id");

if (id && id.length > 20 && !id.includes("folders") && !seen.has(id)) {

seen.add(id);

const divs = tr.querySelectorAll("div[aria-label]");

for (const div of divs) {

let label = div.getAttribute("aria-label");

if (label && label.includes("Pasta compartilhada")) {

label = label.replace(" Pasta compartilhada", "").trim();

if (label.length > 2) {

results.push({name: label, folder_id: id});

break;

}

}

}

}

});

return results;

}''')

`

Month Label Patterns by Year

• - 2023: 08/2023, 09/2023, etc. (slash format)
• - 2024: 01/2024, 02/2024, etc. (slash format)
• - 2025: janeiro, fevereiro, março, etc. (portuguese month names)
• - 2026: janeiro through junho (portuguese)

DocCorretor Edge Function (migrate-drive) API

• - Action: insert_docs
• - Payload key: documents (NOT docs)
• - Required fields per document: original_name, title, stored_name, client_id, folder_id, uploaded_by
• - folder_id in client_folders: Must be a valid UUID, not the 49-char Google Drive ID. Generate with uuid.uuid4().
• - title field: Required (NOT NULL constraint)

Storage & Permissions

• - Storage base: /var/www/documentos/uploads/5a4ebb41-9e54-47b3-86eb-1c18bc01ad5a/{client_id}/
• - After copying files: chmod -R o+r {client_storage_dir} (or individual files if 600 perms)
• - Uploaded_by: 5a4ebb41-9e54-47b3-86eb-1c18bc01ad5a (Vinicius profile ID)

HTML/CSS Support Files (Not Real Documents)

• - Google Drive HTML exports create _files/ directories with CSS/JS/PNG assets.
• - Files like all.css, bootstrap.min.js, scripts.js.download, data.json, favicon.ico are NOT documents — exclude them.

Migration Script

See /tmp/migrate_month.py — incomplete, browser download not yet implemented. Only month/client discovery works.

Remaining Work

• - Implement browser-based file download (Playwright) for "Proprietário oculto" folders
• - Migrate 2023 (09-12), 2024 (01-12), 2025 (01-12), 2026 (01-06)
• - 08/2023 already migrated (138 docs, 12 clients)