feat(docs): add developer documentation

packages/firestore/.gemini/commands/docs.toml

@@ -0,0 +1,13 @@
+description = "Generate documentation for a high level concept in the codebase"
+prompt = """
+
+Your primary role is to help explain this codebase and update the existing documentation with useful information for future maintainers new to the codebase.
+
+1. **Understand the existing documentation** Read everything in @devdocs/ to get a high level understanding of the concepts and core components of this codebase. Look for any relevant information on {{args.concept}}.
+2. **Perform archeology on the codebase** Read everything in {{args.file_or_folder}} and learn more about {{args.concept}}.
+3. **Verify your understanding of the concept** Stop and give a high level summary on {{args.concept}} is. Ask me questions to ensure you and and I are aligned on the code, the concept, why it exists, and the intended intended use. Create a high level summary of what you learned. Repeat asking me questions and summarizing until I tell you I'm ready to document what you know.
+4. **Think carefully if and how to document this** Think deeply about what you know and if it is worth documenting. If so, think about the best way to document it. Documentation files, code comments, or both. Prefer high level documentation.
+5. **Recommend a documentation plan** Re-read the documentation in @devdocs/ and recommend changes that would be helpful for future maintainers new to the codebase. Changes can include edits to the existing documentation or the creation of new documentation files. Edits to existing files must align to their purpose and level of detail. When creating new files, consider linking to it from existing documentation files for discovery. The content must be high level concepts with minimal references to code. Iterate with me until I'm satisfied with the proposed edits.
+
+Your final output is a recommendation to the documentation.
+"""

packages/firestore/GEMINI.md

@@ -0,0 +1,5 @@
+# Firestore JavaScript SDK
+This project is the official JavaScript SDK for the [Google Cloud Firestore](https://firebase.google.com/docs/firestore) database.
+
+You are an expert in @devdocs/prerequisites.md
+@devdocs/overview.md

packages/firestore/devdocs/architecture.md

@@ -0,0 +1,104 @@
+# SDK Architecture
+
+This document provides a detailed explanation of the Firestore JavaScript SDK's architecture, its core components, and the flow of data through the system.
+
+## Core Components
+
+The SDK is composed of several key components that work together to provide the full range of Firestore features.
+
+![Architecture Diagram](./architecture.png)
+
+*   **API Layer**: The public-facing API surface that developers use to interact with the SDK. This layer is responsible for translating the public API calls into the internal data models and passing them to the appropriate core components.
+*   **Core**:
+    *   **Event Manager**: Acts as a central hub for all eventing in the SDK. It is responsible for routing events between the API Layer and Sync Engine. It manages query listeners and is responsible for raising snapshot events, as well as handling connectivity changes and some query failures.
+    *   **Sync Engine**: The central controller of the SDK. It acts as the glue between the Event Manager, Local Store, and Remote Store. Its responsibilities include:
+        *   Coordinating client requests and remote events.
+        *   Managing a view for each query, which represents the unified view between the local and remote data stores.
+        *   Notifying the Remote Store when the Local Store has new mutations that need to be sent to the backend.
+*   **Local Store**: A container for the components that manage persisted and in-memory data.
+    *   **Remote Table**: A cache of the most recent version of documents as known by the Firestore backend.
+    *   **Mutation Queue**: A queue of all the user-initiated writes (set, update, delete) that have not yet been acknowledged by the Firestore backend.
+    *   **Local View**: A cache that represents the user's current view of the data, combining the Remote Table with the Mutation Queue.
+    *   **Overlays**: A performance-optimizing cache that stores the calculated effect of pending mutations from the Mutation Queue on documents. Instead of re-applying mutations every time a document is read, the SDK computes this "overlay" once and caches it, allowing the Local View to be constructed more efficiently.
+*   **Remote Store**: The component responsible for all network communication with the Firestore backend. It manages the gRPC streams for reading and writing data, and it abstracts away the complexities of the network protocol from the rest of the SDK.
+*   **Persistence Layer**: The underlying storage mechanism used by the Local Store to persist data on the client. In the browser, this is implemented using IndexedDB.
+
+The architecture and systems within the SDK map closely to the directory structure, which helps developers navigate the codebase. Here is a mapping of the core components to their corresponding directories.
+
+*   `src/`:
+    *   `api/`: Implements the **API Layer** for the main SDK.
+    *   `lite-api/`: Implements the **API Layer** for the lite SDK.
+    *   `core/`: Implements the **Sync Engine** and **Event Manager**.
+    *   `local/`: Implements the **Local Store**, which includes the **Mutation Queue**, **Remote Table**, **Local View**, **Overlays** and the **Persistence Layer**.
+    *   `remote/`: Implements the **Remote Store**, handling all network communication.
+
+For a more detailed explanation of the contents of each directory, see the [Code Layout](./code-layout.md) documentation.
+
+## Overview of features
+
+At a high level, all interactions with Firestore can be categorized as either reading or writing data. The SDK provides different mechanisms for these operations, each with distinct guarantees and performance characteristics. There is also a special case of writing data called tansactions detailed below.
+
+
+### Read Operations
+
+There are two fundamental ways to read data from Firestore:
+
+*   **One-Time Reads**: This is for fetching a snapshot of data at a specific moment. It's a simple request-response model. You ask for a document or the results of a query, and the server sends back the data as it exists at that instant.
+
+*   **Real-Time Listeners**: This allows you to subscribe to a document or a query. The server first sends you the initial data and then continues to push updates to your client in real time as the data changes. This is the foundation of Firestore's real-time capabilities.
+
+When a query is executed, the SDK immediately returns data from the local cache, which includes any pending optimistic writes from the **Mutation Queue**. This provides a fast, responsive experience. At the same time, the SDK sends the query to the Firestore backend to fetch the latest version of the documents. When the fresh documents arrive from the backend, the SDK takes these server-authoritative documents and re-applies any pending mutations from the local queue on top of them. It then re-runs the original query against this newly merged data. If the documents still match the query's criteria, they are delivered to the query listener again. This is a common occurrence and means a listener could see an event for the same document twice: first with the cached, optimistic data, and a second time after the backend data is reconciled.
+
+### Write Operations
+
+All data modifications—creates, updates, and deletes—are treated as "writes." The SDK is designed to make writes atomic and resilient.  There are two fundamental ways to write data to Firestore:
+
+*   **One-Time Writes**: When a user performs a write (create, update, or delete), the operation is not sent directly to the backend. Instead, it's treated as a "mutation" and added to the local **Mutation Queue**. The SDK "optimistically" assumes the write will succeed on the backend and immediately reflects the change in the local view of the data, making the change visible to local queries. The SDK then works to synchronize this queue with the backend. This design is crucial for supporting offline functionality, as pending writes can be retried automatically when network connectivity is restored.
+
+*   **Transactions**: For grouping multiple write operations into a single atomic unit, the SDK provides `runTransaction`. Unlike standard writes, transactions do not use the optimistic, offline-capable write pipeline. Instead, they are sent directly to the backend, which requires an active internet connection. This ensures atomicity but means transactions do not benefit from the offline capabilities of the standard write pipeline.
+
+### Data Bundles
+
+A Firestore data bundle is a serialized collection of documents and query results, created on a server using the Firebase Admin SDK. Bundles are used to efficiently deliver a pre-packaged set of data to the client, which can then be loaded directly into the SDK's local cache. This is useful for:
+
+*   **Seeding initial data** for an application, allowing users to have a complete offline experience on their first use.
+*   **Distributing curated datasets** to clients in a single, efficient package.
+
+When a bundle is loaded, its contents are unpacked and stored in the local cache, making the data available for immediate querying without needing to connect to the Firestore backend. For more details, see the [Bundles documentation](./bundles.md).
+
+
+# Data Flow
+
+Here's a step-by-step walkthrough of how data flows through the SDK for a write operation, referencing the core components.
+
+## Write Data Flow
+
+1.  **API Layer**: A user initiates a write operation (e.g., `setDoc`, `updateDoc`, `deleteDoc`).
+2.  **Sync Engine**: The call is routed to the Sync Engine, which wraps the operation in a "mutation".
+3.  **Mutation Queue (in Local Store)**: The Sync Engine adds this mutation to the Mutation Queue. The queue is persisted to the **Persistence Layer** (IndexedDB). At this point, the SDK "optimistically" considers the write successful locally.
+4.  **Local View (in Local Store)**: The change is reflected in the Local View. This is done by creating or updating a cached **Overlay** for the affected document, making the change efficiently available to any active listeners without waiting for backend confirmation.
+5.  **Remote Store**: The Sync Engine notifies the Remote Store that there are pending mutations.
+6.  **Backend**: The Remote Store sends the mutations from the queue to the Firestore backend.
+7.  **Acknowledgement**: The backend acknowledges the write.
+8.  **Mutation Queue (in Local Store)**: The Remote Store informs the Sync Engine, which then removes the acknowledged mutation from the Mutation Queue.
+
+## Read Data Flow (with a Real-Time Listener)
+
+1.  **API Layer**: A user attaches a listener to a query (e.g., `onSnapshot`).
+2.  **Event Manager**: The Event Manager creates a listener and passes it to the Sync Engine.
+3.  **Sync Engine**: The Sync Engine creates a "view" for the query.
+4.  **Local View (in Local Store)**: The Sync Engine asks the Local Store for the current documents matching the query. The Local Store provides these by applying cached **Overlays** on top of the documents to reflect optimistic local changes from the **Mutation Queue**.
+5.  **API Layer**: The initial data from the Local View is sent back to the user's `onSnapshot` callback. This provides a fast, initial result.
+6.  **Remote Store**: Simultaneously, the Sync Engine instructs the Remote Store to listen to the query on the Firestore backend.
+7.  **Backend**: The backend returns the initial matching documents for the query.
+8.  **Remote Table (in Local Store)**: The Remote Store receives the documents and saves them to the Remote Table in the Local Store, overwriting any previously cached versions of those documents.
+9.  **Sync Engine**: The Sync Engine is notified of the updated documents. It re-calculates the query view by combining the new data from the Remote Table with any applicable pending mutations from the **Mutation Queue**.
+10. **API Layer**: If the query results have changed after this reconciliation, the new results are sent to the user's `onSnapshot` callback. This is why a listener may fire twice initially.
+11. **Real-time Updates**: From now on, any changes on the backend that affect the query are pushed to the Remote Store, which updates the Remote Table, triggering the Sync Engine to re-calculate the view and notify the listener.
+
+## Bundle Loading Data Flow
+
+1.  **API Layer**: The user initiates a bundle load via the public API.
+2.  **Sync Engine**: The Sync Engine receives the bundle and begins processing it.
+3.  **Local Store**: The Sync Engine unpacks the bundle and saves its contents (documents and named queries) into the **Local Store**.
+4.  **API Layer**: The user is notified of the progress and completion of the bundle loading operation via the task returned by the API.

packages/firestore/devdocs/build.md

@@ -0,0 +1,3 @@
+# Build Process
+
+This document provides a detailed explanation of the Firestore JavaScript SDK build process for the main and lite packages.

packages/firestore/devdocs/bundles.md

@@ -0,0 +1,58 @@
+# Firestore Data Bundles
+
+This document provides a deep dive into the concept of Firestore data bundles, how they are processed, and how they are used within the SDK.
+
+## What is a Bundle?
+
+A Firestore data bundle is a serialized, read-only collection of documents and named query results. Bundles are created on a server using the Firebase Admin SDK and can be efficiently distributed to clients.
+
+While bundles can be used for several purposes, their primary design motivation is to optimize Server-Side Rendering (SSR) workflows. In an SSR setup, a server pre-renders a page with data from Firestore. This data can be packaged into a bundle and sent to the client along with the HTML. The client-side SDK can then load this bundle and "hydrate" a real-time query with the pre-existing data, avoiding the need to re-fetch the same documents from the backend. This results in a significant performance improvement and cost savings.
+
+## Primary Use Case: Server-Side Rendering (SSR) Hydration
+
+The main workflow for bundles is as follows:
+
+1.  **Server-Side:** A server fetches data from Firestore to render a page.
+2.  **Bundling:** The server packages the fetched documents and the corresponding query into a bundle.
+3.  **Transmission:** The bundle is embedded in the HTML page sent to the client.
+4.  **Client-Side:** The client-side JavaScript calls `loadBundle()` to load the data from the bundle into the SDK's local cache.
+5.  **Hydration:** The client then attaches a real-time listener to the same query that was bundled. The SDK finds the query results in the local cache and immediately fires the listener with the initial data, avoiding a costly roundtrip to the backend.
+
+## Other Benefits and Use Cases
+
+Beyond SSR hydration, bundles offer several other advantages:
+
+*   **Enhanced Offline Experience:** Bundles can be shipped with an application's initial assets, allowing users to have a more complete offline experience from the first launch, reducing the need to sync every document individually.
+*   **Efficient Data Distribution:** They provide an efficient way to deliver curated or static datasets to clients in a single package. For instance, an application could bundle a list of popular items or configuration data.
+
+## The Loading Process
+
+The process of loading a bundle into the Firestore SDK is initiated by the `loadBundle()` method. This method returns a `LoadBundleTask`, which allows the developer to track the progress of the loading operation.
+
+Here's a step-by-step walkthrough of what happens when `loadBundle()` is called:
+
+1.  **`loadBundle()` called:** The developer calls `loadBundle()` with the bundle data (as a `ReadableStream` or `ArrayBuffer`).
+2.  **`LoadBundleTask` created:** A `LoadBundleTask` is created and returned to the developer. This task acts as a `Promise` and also provides progress updates.
+3.  **`BundleLoader` initiated:** Internally, a `BundleLoader` is created to process the bundle.
+4.  **Bundle processing:** The `BundleLoader` reads the bundle element by element. The bundle is a sequence of JSON objects, each representing a metadata element, a named query, or a document.
+5.  **Data caching:** As the `BundleLoader` processes the bundle, it saves the data to the local store:
+    *   **Bundle Metadata:** The bundle's metadata is saved to the `BundleCache`. This is used to track which bundles have been loaded.
+    *   **Named Queries:** Named queries are saved to the `BundleCache`.
+    *   **Documents:** Documents are saved to the `RemoteDocumentCache`.
+6.  **Progress updates:** The `LoadBundleTask` is updated with progress information (e.g., bytes loaded, documents loaded) as the `BundleLoader` processes the bundle.
+7.  **Completion:** Once the `BundleLoader` has finished processing the bundle, the `LoadBundleTask` is marked as complete.
+
+## Error Handling
+
+Errors can occur during the bundle loading process for a variety of reasons, such as a malformed bundle or a storage issue.
+
+When an error occurs, the `LoadBundleTask` is put into an `'Error'` state. The error is surfaced to the developer in two ways:
+
+*   **Promise rejection:** The `LoadBundleTask`'s promise is rejected with a `FirestoreError`.
+*   **`onProgress` observer:** If an `error` callback is provided to the `onProgress` method, it will be called with the `FirestoreError`.
+
+## Interacting with Bundled Data
+
+Once a bundle has been loaded, the data it contains is available for querying. If the bundle included named queries, you can use the `getNamedQuery()` method to retrieve a `Query` object, which can then be executed.
+
+When a named query is executed, the Firestore SDK will first attempt to fulfill the query from the local cache. If the results for the named query are available in the cache (because they were loaded from a bundle), they will be returned immediately, without a server roundtrip.

packages/firestore/devdocs/code-layout.md

@@ -0,0 +1,23 @@
+# SDK Code Layout
+
+This document explains the code layout in this repository. It is closely related to the [architecture](./architecture.md).
+
+*   `src/`: Contains the source code for the main `@firebase/firestore` package.
+    *   `api/`: Implements the **API Layer** for the main SDK.
+    *   `lite-api/`: Contains the entry point of for the lite SDK.
+    *   `core/`: Contains logic for the **Sync Engine** and **Event Manager**.
+    *   `local/`: Contains the logic the **Local Store**, which includes the **Mutation Queue**, **Remote Table**, **Local View**, **Overlays**, and the **Persistence Layer**.
+    *   `remote/`: Contains the logic for the **Remote Store**, handling all network communication.
+    *   `model/`: Defines the internal data models used throughout the SDK, such as `Document`, `DocumentKey`, and `Mutation`. These models are used to represent Firestore data and operations in a structured way.
+    *   `platform/`: Contains platform-specific code to abstract away the differences between the Node.js and browser environments. This includes things like networking, storage, and timers. This allows the core logic of the SDK to be platform-agnostic.
+    *   `protos/`: Contains the Protocol Buffer (`.proto`) definitions that describe the gRPC API surface of the Firestore backend. These files are used to generate the client-side networking code.
+*   `lite/`: Defines the entrypoint code for the `@firebase/firestore/lite` package.
+*   `test/`: Contains all unit and integration tests for the SDK. The tests are organized by component and feature, and they are essential for ensuring the quality and correctness of the code.
+*   `scripts/`: Contains a collection of build and maintenance scripts used for tasks such as bundling the code, running tests, and generating documentation.
+
+TODO: Add more detailed information as appropriate on each folder
+
+TODO: Mention critical entry points
+    - `package.json` for packages and common commands. Go to [build.md](./build.md) for details
+    - rollup configs for main and lite sdks. Go to [build.md](./build.md) for details
+    - tests entry points. Go to [testing.md](./testing.md) for details