1. Integration
1.1. AzureFunction
1.1.1. Create
1.1.1.1. Trigger:CosmosDBTrigger
1.1.1.2. CosmosDBAccount
1.1.1.3. ComsmosDatabase
1.1.1.4. CosmosCollection
1.1.1.5. LeaseCollection
2. DataModeling
2.1. Embed fields when
2.1.1. Read or updated together:
2.1.2. 1:1 relationship
2.1.2.1. Customer:CustomerPassword
2.1.3. 1:Few relationship
2.1.3.1. Customer:CustomerAddresses
2.2. Reference (separate doc) when
2.2.1. Read or updated independently
2.2.2. 1:Many relationship
2.2.3. Many:Many relationship
3. Container
3.1. Physical Partitions
3.1.1. P1
3.1.1.1. Each Physical partition has max 50GB storage and 10K RU's
3.1.2. P2
3.1.3. P3
3.1.3.1. LP1
3.1.3.2. LP1
4. DataModeling
4.1. Data modeling based on query with aggregates
4.1.1. top 10 customers by the number of sales orders
4.1.1.1. he solution here is to denormalize the aggregate value in a new property, salesOrderCount, in the customer document.
4.1.1.1.1. Azure Cosmos DB supports transactions when the data sits within the same logical partition
4.2. If two entities have same partitionKey and similar access patterns then it can be stored together in single container "CustomerSalesOrder"
4.2.1. All the orders of Customer.
4.2.1.1. SELECT * FROM Order O WHERE O.customerId = "A" and o.type ="Order"
4.2.2. Customer Details
4.2.2.1. SELECT * FROM Customer C WHERE C.CustomerID= "A" and c.type="Customer"
4.2.2.1.1. { Id:'A',type:'customer',customerId:"A" } , { Id:"O1",type:"Order",customerId:"A }
4.3. ReferentialIntegrity
4.3.1. Use Change feed
5. Indexing
5.1. Read heavy
5.1.1. index seek
5.1.1.1. =, IN
5.1.2. index scan
5.1.2.1. RANGE
5.1.3. full scan
5.1.3.1. no filter
5.2. write heavy
5.2.1. IndexingMode: None to disable
5.2.2. exclude all and include very few
5.3. SDK index debugging performance and index recommendations
5.3.1. QueryRequestOptions options = new() { PopulateIndexMetrics = true }; Console.WriteLine(response.IndexMetrics);
6. Integrated cache
6.1. Enable Dedicated Gateway
6.1.1. SDK
6.1.1.1. Use gateway connString
6.1.1.2. CosmosClientOptions options = new() { ConnectionMode = ConnectionMode.Gateway };
6.1.1.3. QueryRequestOptions queryOptions = new() { ConsistencyLevel = ConsistencyLevel.Eventual }; //Or Session
6.1.1.3.1. Staleness (Expiry)
7. Region
8. Unique DNS Name
9. Capacity
9.1. Throughput provisioned
9.1.1. unlimited storage, predictable traffic, req consumed > 66 % of hours per month
9.2. Serverless
9.2.1. Max 50GB storage, unpredicted traffic,request consumed < 66% of hours per month
10. DataBase
10.1. Provision throughput
10.1.1. Autoscale
10.1.1.1. Maximum RU's
10.1.1.1.1. Charge will be 10% of the max RU when its not used
10.1.2. Manual
10.1.2.1. Fixed RU's
10.1.2.1.1. Min 400 RU's
11. Container
11.1. PartitionKeyPath
11.2. Provision Dedicated Throughput
11.2.1. Autoscale
11.2.2. Manual
11.3. Unique Key
11.4. Analytical store for Synapse Link integration
11.5. Change Feed
11.5.1. Monitored container
11.5.1.1. This container is monitored for any insert or update operations
11.5.2. Lease container
11.5.2.1. serves as a storage mechanism to manage state across multiple change feed consumers (clients)
11.5.3. Host
11.5.3.1. client application instance that listens for and reacts to changes from the change feed
11.5.3.1.1. ChangeFeedProcessor
11.5.3.1.2. ChangeFeedEstimator
11.5.4. Delegate
11.5.4.1. code within the client application that will implement business logic for each batch of changes
11.5.4.1.1. ChangesHandler<Product> changeHandlerDelegate = async ( IReadOnlyCollection<Product> changes, CancellationToken cancellationToken ) => { foreach(Product product in changes) { await Console.Out.WriteLineAsync($"Detected Operation:\t[{product.id}]\t{product.name}"); // Do something with each change } };
11.5.4.1.2. ChangesEstimationHandler changeEstimationDelegate = async ( long estimation, CancellationToken cancellationToken ) => { // Do something with the estimation };
11.6. SDK
11.6.1. CosmosClient client = new("ConnString")
11.6.1.1. db = client.CreateDatbaseIfNotExistsAsync("dbName");
11.6.1.1.1. container = client.CreateContainerIfNotExistsAsync("Name","PartitionKeyPath","RU")
11.6.2. Logging
11.6.2.1. CosmosClientBuilder b = new(new LogHander()) // : RequestHanlder =>override SendAsync // client = b.build();
12. CosmosClientOptions
13. Troubleshooting 429 Request too large exception
13.1. Azure Cosmos DB Insight
13.1.1. Request tab
13.1.1.1. Total Request by Status Code if worklaod < 5%
13.1.1.1.1. Yes
13.1.1.1.2. No
14. Status codes and meaning
14.1. Actions
14.1.1. Create a Document
14.1.1.1. 400 : Bad Req
14.1.1.1.1. The JSON body is invalid.
14.1.1.2. 403 : Forbidden
14.1.1.2.1. The operation couldn't be completed because the storage limit of the partition has been reached.
14.1.1.3. 409 : Conflict
14.1.1.3.1. The id provided for the new document has been taken by an existing document.
14.1.1.4. 413 : Entity too large
14.1.1.4.1. The document size in the request exceeded the allowable document size.
14.1.2. List documents using ReadFeed
14.1.2.1. 400 : Bad Req
14.1.2.1.1. The override set in x-ms-consistency-level is stronger than the one set during account creation. For example, if the consistency level is Session, the override can't be Strong or Bounded.
14.1.3. Get document by point read
14.1.3.1. 304 : Not modified
14.1.3.1.1. The document requested wasn't modified since the specified eTag value in the If-Match header. The service returns an empty response body.
14.1.3.2. 400 : Bad Req
14.1.3.2.1. The override set in x-ms-consistency-level is stronger than the one set during account creation. For example, if the consistency level is Session, the override can't be Strong or Bounded.
14.1.4. Patch
14.1.4.1. 412: PreCondition Failed
14.1.5. Exception Codes
14.1.5.1. 408: Request timed out
14.1.5.1.1. The operation did not complete within the allotted amount of time. This code is returned when a stored procedure, trigger, or UDF (within a query) does not complete execution within the maximum execution time.
14.1.5.2. 429:Too many requests
14.1.5.2.1. The collection has exceeded the provisioned throughput limit. Retry the request after the server specified retry after duration. For more information, see request units.
15. Azure Cosmos DB Explorer to be able to sign in using RBAC
15.1. enable the property ?feature.enableAadDataPlane=true
16. Always Encrypt Very sensitive details
16.1. Setup Azure Key Vault
16.1.1. 1.Create a new key in KV. 2.Create AD app and generate client secret 3.Geive permissions (Get, List, Unwrap Key, Wrap Key, **Verify, and Sign) in KV access policy to the AD app.
16.2. Initialize the SDK
16.2.1. var tokenCredential = new ClientSecretCredential( "<aad-app-tenant-id>", "<aad-app-client-id>", "<aad-app-secret>"); var keyStoreProvider = new AzureKeyVaultKeyStoreProvider(tokenCredential); var client = new CosmosClient("<connection-string>") .WithEncryption(keyStoreProvider);
16.3. Create a data encryption key
16.3.1. var database = client.GetDatabase("my-database"); await database.CreateClientEncryptionKeyAsync( "my-key", DataEncryptionKeyAlgorithm.AEAD_AES_256_CBC_HMAC_SHA256, new EncryptionKeyWrapMetadata( keyStoreProvider.ProviderName, "akvKey", "https://<my-key-vault>.vault.azure.net/keys/<key>/<version>"));
16.4. Create a container with encryption policy
16.4.1. var path1 = new ClientEncryptionIncludedPath { Path = "/property1", ClientEncryptionKeyId = "my-key", EncryptionType = EncryptionType.Deterministic.ToString(), EncryptionAlgorithm = DataEncryptionKeyAlgorithm.AEAD_AES_256_CBC_HMAC_SHA256.ToString() }; var path2 = new ClientEncryptionIncludedPath { Path = "/property2", ClientEncryptionKeyId = "my-key", EncryptionType = EncryptionType.Randomized.ToString(), EncryptionAlgorithm = DataEncryptionKeyAlgorithm.AEAD_AES_256_CBC_HMAC_SHA256.ToString() }; await database.DefineContainer("my-container", "/partition-key") .WithClientEncryptionPolicy() .WithIncludedPath(path1) .WithIncludedPath(path2) .Attach() .CreateAsync();
16.5. Filter queries on encrypted properties
16.5.1. var queryDefinition = container.CreateQueryDefinition( "SELECT * FROM c where c.property1 = @Property1"); await queryDefinition.AddParameterAsync( "@Property1", 1234, "/property1");
17. AzCommands
17.1. Create Account: az cosmosdb create \ --name '<account-name>' \ --resource-group '<resource-group>'
17.2. Create Account with Free tier and "Strong" Consistency az cosmosdb create \ --name '<account-name>' \ --resource-group '<resource-group>' \ --default-consistency-level 'strong' \ --enable-free-tier 'true'
17.3. Create Account with Regions az cosmosdb create \ --name '<account-name>' \ --resource-group '<resource-group>' \ --locations regionName='eastus'
17.4. Create database with az cosmosdb sql az cosmosdb sql database create \ --account-name '<account-name>' \ --resource-group '<resource-group>' \ --name '<database-name>'
17.5. Create container az cosmosdb sql container create \ --account-name '<account-name>' \ --resource-group '<resource-group>' \ --database-name '<database-name>' \ --name '<container-name>' \ --throughput '400' \ --partition-key-path '<partition-key-path-string>'
17.6. Create Container with indexing policy az cosmosdb sql container create \ --account-name '<account-name>' \ --resource-group '<resource-group>' \ --database-name '<database-name>' \ --name '<container-name>' \ --partition-key-path '<partition-key-path-string>' \ --idx '@.\policy.json' \ --throughput '400'
17.7. Update Container throughput az cosmosdb sql container throughput update \ --account-name '<account-name>' \ --resource-group '<resource-group>' \ --database-name '<database-name>' \ --name '<container-name>' \ --throughput '1000'
17.8. Update database throughput az cosmosdb sql database throughput update \ --account-name '<account-name>' \ --resource-group '<resource-group>' \ --name '<database-name>' \ --throughput '4000'
17.9. Update throughput type az cosmosdb sql container throughput migrate \ --account-name '<account-name>' \ --resource-group '<resource-group>' \ --database-name '<database-name>' \ --name '<container-name>' \ --throughput-type 'autoscale'
17.10. Update max throughput az cosmosdb sql container throughput update \ --account-name '<account-name>' \ --resource-group '<resource-group>' \ --database-name '<database-name>' \ --name '<container-name>' \ --max-throughput '5000'
17.11. Show min throughput az cosmosdb sql container throughput show \ --account-name '<account-name>' \ --resource-group '<resource-group>' \ --database-name '<database-name>' \ --name '<container-name>' \ --query 'resource.minimumThroughput' \ --output 'tsv'
17.12. Add account regions with failover az cosmosdb update \ --name '<account-name>' \ --resource-group '<resource-group>' \ --locations regionName='eastus' failoverPriority=0 isZoneRedundant=False \ --locations regionName='westus2' failoverPriority=1 isZoneRedundant=False \ --locations regionName='centralus' failoverPriority=2 isZoneRedundant=False
17.13. Enable automatic failover az cosmosdb update \ --name '<account-name>' \ --resource-group '<resource-group>' \ --enable-automatic-failover 'true'
17.14. Change failover priorities az cosmosdb failover-priority-change \ --name '<account-name>' \ --resource-group '<resource-group>' \ --failover-policies 'eastus=0' 'centralus=1' 'westus2=2'
17.15. Enable multi-region write az cosmosdb update \ --name '<account-name>' \ --resource-group '<resource-group>' \ --enable-multiple-write-locations 'true'
17.16. Remove account regions az cosmosdb update \ --name '<account-name>' \ --resource-group '<resource-group>' \ --locations regionName='eastus' failoverPriority=0 isZoneRedundant=False \ --locations regionName='westus2' failoverPriority=1 isZoneRedundant=False
17.17. Initiate failovers hanging the region with priority = 0 will trigger a manual failover for an Azure Cosmos account. az cosmosdb failover-priority-change \ --name '<account-name>' \ --resource-group '<resource-group>' \ --failover-policies 'westus2=0' 'eastus=1'
18. ARM Template for creating Acc/Db/Container
18.1. { "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json# ", "contentVersion": "1.0.0.0", "resources": [ { "type": "Microsoft.DocumentDB/databaseAccounts", "apiVersion": "2021-05-15", "name": "[concat('csmsarm', uniqueString(resourceGroup().id))]", "location": "[resourceGroup().location]", "properties": { "databaseAccountOfferType": "Standard", "locations": [ { "locationName": "westus" } ] } }, { "type": "Microsoft.DocumentDB/databaseAccounts/sqlDatabases", "apiVersion": "2021-05-15", "name": "[concat('csmsarm', uniqueString(resourceGroup().id), '/cosmicworks')]", "dependsOn": [ "[resourceId('Microsoft.DocumentDB/databaseAccounts', concat('csmsarm', uniqueString(resourceGroup().id)))]" ], "properties": { "resource": { "id": "cosmicworks" } } }, { "type": "Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers", "apiVersion": "2021-05-15", "name": "[concat('csmsarm', uniqueString(resourceGroup().id), '/cosmicworks/products')]", "dependsOn": [ "[resourceId('Microsoft.DocumentDB/databaseAccounts', concat('csmsarm', uniqueString(resourceGroup().id)))]", "[resourceId('Microsoft.DocumentDB/databaseAccounts/sqlDatabases', concat('csmsarm', uniqueString(resourceGroup().id)), 'cosmicworks')]" ], "properties": { "options": { "throughput": 400 }, "resource": { "id": "products", "partitionKey": { "paths": [ "/categoryId" ] } } } } ] }
18.2. Bicep Template resource Account 'Microsoft.DocumentDB/databaseAccounts@2021-05-15' = { name: 'csmsbicep${uniqueString(resourceGroup().id)}' location: resourceGroup().location properties: { databaseAccountOfferType: 'Standard' locations: [ { locationName: 'westus' } ] } } resource Database 'Microsoft.DocumentDB/databaseAccounts/sqlDatabases@2021-05-15' = { parent: Account name: 'cosmicworks' properties: { options: { } resource: { id: 'cosmicworks' } } } resource Container 'Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers@2021-05-15' = { parent: Database name: 'customers' properties: { resource: { id: 'customers' partitionKey: { paths: [ '/regionId' ] } } } }
19. Deploy to ResourceGroup
19.1. az deployment group create \ --resource-group '<resource-group>' \ --template-file '.\template.json'
19.1.1. az deployment group create \ --resource-group '<resource-group>' \ --template-file '.\template.bicep'
19.2. az deployment group create \ --resource-group '<resource-group>' \ --name '<deployment-name>' \ --template-file '.\template.json'
19.3. az deployment group create \ --resource-group '<resource-group>' \ --template-file '.\template.json' \ --parameters name='<value>'
19.4. az deployment group create \ --resource-group '<resource-group>' \ --template-file '.\template.json' \ --parameters '@.\parameters.json'
20. Steps to access cosmos connestr from key vault
20.1. Store cosmos constr to Key vault as secret
20.2. Create azure app service and create managed identity
20.3. Provide access in KeyVault using access policy for the app service identity
20.4. Create web application that has logic to connect key vault and get secret key and communicate cosmos db.
20.4.1. AzureServiceTokenProvider tok = new AzureServiceTokenProvider
21. Limits RBAC
21.1. You can create up to 100 role definitions and 2,000 role assignments per Azure Cosmos DB account.
21.2. You can only assign role definitions to Azure AD identities belonging to the same Azure AD tenant as your Azure Cosmos DB account.
21.3. Azure AD group resolution isn't currently supported for identities that belong to more than 200 groups.
21.4. The Azure AD token is currently passed as a header with each individual request sent to the Azure Cosmos DB service, increasing the overall payload size.
22. Disable Access Keys using Arm template
22.1. disableLocalAuth = true
23. Built in roles RBAC
23.1. DocumentDB Account Contributor
23.1.1. Can manage Azure Cosmos DB accounts
23.2. Cosmos DB Account Reader
23.2.1. Can read Azure Cosmos DB account data.
23.3. Cosmos Backup Operator
23.3.1. Can submit a restore request from the Azure portal for a periodic backup-enabled database or a container. Can modify the backup interval and retention on the Azure portal. Can't access any data or use Data Explorer.
23.4. CosmosRestoreOperator
23.4.1. Can perform a restore action for an Azure Cosmos DB account with continuous backup mode
23.5. Cosmos DB Operator
23.5.1. Can provision Azure Cosmos accounts, databases, and containers. Can't access any data or use Data Explorer.
24. Restore
24.1. Only by support request
24.1.1. Eligible roles: Azure Cosmos DB owner, contributor, or have the Cosmos DB Operator (CosmosdbBackupOperator)
24.1.2. Other than "Basic" Plan
24.1.3. Other than "Basic" Plan
24.2. Settings that are not restored
24.2.1. VNET access control lists
24.2.2. Stored procedures, triggers, and user-defined functions
24.2.3. Multi-region settings
25. Backup
25.1. Periodic
25.1.1. Frequency: default value 4 hrs max : 24 hrs
25.1.1.1. Redundancy
25.1.1.1.1. Geo (Default)
25.1.1.1.2. Zone
25.1.1.1.3. local
25.1.1.1.4. can be altered even after account creation
25.1.2. Retention: Default value: 8 hrs Max:30 days
25.2. continuous (incur costs)
25.2.1. Retention 30 days
25.2.1.1. Redundancy
25.2.1.1.1. local (default)
25.2.1.1.2. Zone
25.2.1.1.3. redundancy can't be updated further
25.2.1.1.4. Restore
25.2.2. Not supported for below cases
25.2.2.1. customer managed certificate used
25.2.2.2. Multi-region write accounts
25.2.2.3. ou can't restore an account into a region where the source account did not exist
25.2.2.4. The retention period is 30 days and can't be changed
25.2.2.5. Can't modify or delete IAM policies when restore is in progress
25.2.2.6. Point in time restore always restores to a new Azure Cosmos DB account
25.2.2.7. Collection's consistent indexes may still be rebuilding after completing the restore.
25.2.2.8. Azure Synapse Link and continuous backup mode can't coexist in the same database account
26. monitor server side metrics
26.1. Azure Monitor to monitor metrics Capturing every min Retention 30 days
26.1.1. throughput
26.1.2. storage availability
26.1.3. latency
26.1.4. consistency
26.1.5. system level metrics
26.2. Azure Monitor to monitor diagnostic logs
26.2.1. Telemetries like events and traces are stored as logs ex. throughput change
26.2.1.1. store logs in
26.2.1.1.1. AzureDiagnostics table
26.2.1.1.2. AzureActivity
26.3. The Azure Cosmos DB portal Retention 7 days
26.3.1. All Azure monitor logs
26.4. he Cosmos DB NoSQL API SDKs to programmatically monitor the account
27. logs
27.1. LogAnalytics
27.1.1. AzureDiagnostics
27.1.1.1. AzureDiagnostics | where TimeGenerated >= ago(1h) | where ResourceProvider=="MICROSOFT.DOCUMENTDB" and Category=="DataPlaneRequests"
27.1.2. Resource-specific
27.1.2.1. CDBDataPlaneRequests | where TimeGenerated >= ago(1h)