What are Salesforce's Big Objects?
Salesforce Big Objects are a type of data storage mechanism within the Salesforce platform designed for handling and storing large volumes of data.
We all know that objects in Salesforce are used to manage and manipulate our data. You’re probably pretty familiar with standard objects, custom objects, and external objects. But if you have to manage an enormous amount of data then that’s where you’ll want to leverage Salesforce Big Objects. These objects manage billions of records of data without compromising performance. Salesforce Big Objects allow you to source massive datasets from external systems into a big object, for a complete view of your customers. The greatest benefit of Big Objects is that it does not count against the data storage limit.
Clients and external systems can use a standard set of APIs to access the data of Big Objects. Big Objects are available for both Salesforce Classic and Lightning Experience.
Available in: Enterprise, Performance, Unlimited, and Developer Editions for up to 1 million records
They are typically used for scenarios where you need to manage and analyze data at scale. Here are some key characteristics and use cases of Salesforce Big Objects.
Large Data Volumes: Big Objects are designed to handle massive amounts of data that might be impractical to manage with traditional custom objects in Salesforce.
Data Archiving: They are often used for data archiving purposes, allowing you to store historical data, audit logs, or other records that are not frequently accessed but need to be retained for compliance or reporting reasons.
High-Performance Queries: Big Objects are optimized for performance when querying large datasets. They use asynchronous SOQL (Salesforce Object Query Language) queries, making it efficient to work with large data volumes.
No Field-Level Security: Unlike traditional custom objects, Big Objects don't have field-level security, which means that all fields within a Big Object are treated as accessible by all users with access to the object.
Data Retention Policies: You can define data retention policies for Big Objects, allowing you to automatically delete records that are no longer needed based on certain criteria.
Customizable Schema: You can define the schema of Big Objects, including custom fields and indexes, to suit your specific use case.
Async Processing: Data manipulation and data loading into Big Objects are typically done asynchronously. This means that you submit jobs to Salesforce to perform operations on Big Objects, and they are processed in the background.
Storage-Based Pricing: Salesforce charges for the storage space used by Big Objects, and the pricing can vary based on your Salesforce edition and the amount of data stored.
Limitations: While Big Objects are designed for large data volumes, they come with some limitations. For example, they do not support certain features like triggers, workflow rules, or standard object relationships. Custom Apex code is often needed to work with them effectively.
How to Create a Big Object:
Find big objects in the quick find box the list of the big objects is available there you can also create new big objects from there.
In the big object, there is only limited data types are available. (Lookup Relationship, Date/Time, Email, Number, Phone, Text, Text Area (Long), URL)
What is an index in a big object why do we need to define the index of a field?
The index defines the composite primary key for a custom big object and is used for querying and filtering the big object data.
The fields defined in your index should be the fields that will be most relevant to your queries. Assign the field you will use most frequently in a query filter to the first position in your index. You can also use only specific comparison operators, depending on the field’s position in your query.
Creating big objects through metadata API:
To create the big objects from Metadata API we need an object file that contains the following:
Object Files: Create a file that contains definitions of big objects, fields, and indexes.
PermissionSet/Profile Files: This is not the required file but this file is used to assign the object and field permissions. This grants big object access to the user because by default big objects are restricted.
Package File: A file for the deployment of metadata definitions and permission.
<?xml version="1.0" encoding="UTF-8"?>
<CustomObject xmlns="http://soap.sforce.com/2006/04/metadata">
<deploymentStatus>Deployed</deploymentStatus>
<fields>
<fullName>Start\_Location\_Lat\_\_c</fullName>
<label>Start Location Latitude</label>
<required>false</required>
<type>Number</type>
<scale>4</scale>
<precision>7</precision>
<unique>false</unique>
</fields>
<fields>
<fullName>Start\_Location\_Long\_\_c</fullName>
<label>Start Location Longitude</label>
<required>false</required>
<type>Number</type>
<scale>4</scale>
<precision>7</precision>
<unique>false</unique>
</fields>
<fields>
<fullName>Start\_Time\_\_c</fullName>
<label>Start Time</label>
<required>true</required>
<type>DateTime</type>
<unique>false</unique>
</fields>
<fields>
<fullName>End\_Time\_\_c</fullName>
<label>End Time</label>
<required>false</required>
<type>DateTime</type>
<unique>false</unique>
</fields>
<fields>
<fullName>Service\_Type\_\_c</fullName>
<label>Service Type</label>
<length>16</length>
<required>false</required>
<type>Text</type>
<unique>false</unique>
</fields>
<fields>
<fullName>Rider\_Account\_\_c</fullName>
<label>Rider Account</label>
<length>16</length>
<required>true</required>
<type>Text</type>
<unique>false</unique>
</fields>
<fields>
<fullName>Rider\_Rating\_\_c</fullName>
<label>Rider Rating</label>
<required>false</required>
<type>Number</type>
<scale>1</scale>
<precision>2</precision>
<unique>false</unique>
</fields>
<indexes>
<fullName>Rider\_History\_Index</fullName>
<label>Rider History Index</label>
<fields>
<name>Rider\_Account\_\_c</name>
<sortDirection>DESC</sortDirection>
</fields>
<fields>
<name>Start\_Time\_\_c</name>
<sortDirection>DESC</sortDirection>
</fields>
</indexes>
<label>Rider History</label>
<pluralLabel>Rider Histories</pluralLabel>
</CustomObject>
File location (src->objects):
How to insert a record in the big object.
Archive_Lead__b obj = new Archive_Lead__b();
obj.Account_Lookup__c ='0019000001ukhMF';
obj.Date_Time__c=System.now();
obj.MyTextField__c='hello';
obj.Number_Field__c=10;
database.insertImmediate(obj)
Difference Between Custom Object And Big Object
Let understand the difference between custom object and big object in Salesforce.
Conclusion
When designing any system, it’s always important to consider the volume. As a developer, data storage is a common issue you will run into. However big objects help us to maintain an enormous amount of data without hindering the performance of the system. Big objects provide us with the flexibility to store large amounts of data without worrying about storage capacity. Salesforce Big Objects can help businesses and developers manage large amounts of data.
