One factor to consider while evaluating location data for purchase is data delivery. Storage costs for big data sets can be substantial, so it is important to get the delivery right. The method of data delivery should take your unique use case into account, as well as the platforms you use to work with the data.
Before making a purchase decision, you should ask your vendor if they are flexible enough to use a delivery method that is in line with your requirements.
In this post, we will discuss the various methods of data delivery and how Quadrant delivers data.
Data Storage
Quadrant usually recommends Amazon S3 for easy and efficient data transfer. Data is stored as objects within resources called “buckets”, and a single object/file can be up to 5 terabytes in size.
Data Delivery Modes
• Batch Delivery: Deliver data on a daily basis.
• Streaming Delivery: Deliver data in real-time, every 5-10 minutes.
Organizations either use cloud solutions (from Amazon, Google, Microsoft, etc.) or on-premise storage systems to store and process data. Based on their existing infrastructure, there are various methods to acquire data from Quadrant.
Amazon S3 Data Push / Cross account bucket access
If you are an AWS customer, you can receive data either in batch or streaming mode by simply setting set a few S3 policies. Quadrant can directly access your S3 bucket and push the data.
This is the most preferred and recommended option in terms of ease, convenience, and speed.
Learn more about this integration and policy requirements.
Data Pull by Client (AWS / GCP / Azure)
Data Pull refers to clients pulling the data from Quadrant’s Amazon S3 bucket. This method can also work with any of the data delivery modes – bulk, batch, or streaming.
In this method, you can use any of the leading cloud services – AWS, GCP, or Azure - to receive data. We will create a new IAM (Identity and Access Management) entity and share the bucket details with you. Using the IAM users’ Access Key ID, Secret Access Key, and bucket name, you can pull the data.
File Transfer Protocols
The FTP / SFTP method can only be used for batch delivery. Although this method is not our primary preference, customers can still use it if they do not have a cloud-based system and the daily data size is less than a few hundred MBs.
To receive data via this method, you will need to provide us the hostname, username, password information for your FTP / SFTP server. Quadrant can then connect to their server and push the data on a daily basis.
Quadrant offers production-ready data that is normalized, deduplicated, and ready to be ingested by the customers’ systems and platforms. Unlike most data providers, Quadrant can accommodate your desired data delivery method in the evaluation phase so that you can accurately test the data and make an informed decision.
Have questions about one of the delivery methods stated above or another method you are exploring and interested in? We are happy to help.
Contact our Data Consultants
