Knowledge Base
  • Introduction
  • Events and Actions
  • Action Panel
  • sending email
  • calling a SQL action from a client side js action
  • Accessing to translations form a server
  • Executing SQL statements from within an action
  • How to invoke a generic SQL statement defined through a SQL action
  • How to show a message dialog
  • checking for "undefined" values
  • How to add spaces to the right of a text
  • How to create a docx report and show it on the web browser Enterprise Edition only
  • How to get or set a value from the graphics control
  • How to invoke a generic SQL query defined through a business component
  • How to remove spaces to the left and right of a text
  • How to support multiple themes in a single application, accoding to a rule
  • How to set content to a Google Map linked to a grid or form
  • How to replace all occurences of a pattern from a text
  • Utility methods
  • Link auto login
  • Creation of a link for the first access of a new user without give the user a password and forcing
  • Forgot password
  • setting up default values from values coming from a filter panel
  • identifing the modified record after the alteration
  • enabling/disabling checkboxes in a grid
  • Filtering a Lookup
  • formatting a column
  • using checkboxes to select rows in grid
  • showing a summary row in grid
  • Disabling a toolbar button
  • Configuring grid exports
  • Adding filter conditions to a grid
  • Filtering the grid content from a tree
  • Filtering the tree content, starting from a filter panel linked to a grid
  • collapsing a panel
  • validating a lookup
  • accessing the authorizations set for a specific grid
  • How to design a web service
  • How to remotelly invoke an action or business component or perform a write operation through a Restf
  • how to feed a grid from a JS business component
  • converting a JS object to a JSON string
  • executing a query
  • passing parameters to a server side JS action
  • return value
  • scheduling and frequency
  • finding the right filter panel
  • checking out if a component has been defined
  • Deploying an application
  • Enquiring a table belonging to the Platform repository
  • Adding a where clause to a business component linked to grid
  • Integrating Mailchimp lists
  • Formatting a number as a currency value to use it inside an email template
  • sending email from a template
  • How to send an email
  • Error 'smtpHost' is empty
  • Linking two windows
  • How to open manually a window from another window
  • How to open manually a popup window
  • How to hide a panel in a window dinamically
  • How to manage folder panels
  • How to manage card panels
  • Predefined variables supported by Platform
  • Accessing the application parameters
  • Application Log
  • How to design a web service
  • How to import java classes in server
  • How to import java classes in server
  • How to dynamically set a value on a combo
  • 4WS.Platform
  • How to listen to events in a mobile HTML panel
  • Issues with HTTPS requests
  • How to manage row totals in grid
  • How to send to the UI a notification to execute code automatically
  • How to filter a chart by date interval
  • How to filter a grid by date interval
  • How to read a text or csv file and save data on the database
  • How to write text or csv files
  • Reading an xls file stored in the specified path
  • How to create a report with Jasper Report
  • How to customize the alert message content
  • Setting up a cluster
  • Uploading and downloading files
  • How to listen to user definition changes
  • How to auto-show a window from login
  • How to manage encrypted fields
  • How to change CSS settings for a grid row
  • Customizing a Tree Panel
  • How to execute complex queries on Google Datastore
  • Theme customization
  • Retrieve and send the log of a mobile app
  • Import Roles and Users
  • How to synchronize multiple Form panels in the same window
  • Anchor buttons
  • Properties of subpanels
  • Bulk import
  • How to display the data not found message in a grid
  • How to setup an LDAP based authentication
  • How to synchronize data from Datastore to BigQuery
  • How to synchronize data from Datastore to Google Spanner
  • How to synchronize data from Datastore to CloudSQL
  • Scrollable form list
  • How to setup SAML authentication
  • How to export data from BigQuery in streaming
  • Update Google Spreadsheet
  • How to setup OAuth2 authentication
Powered by GitBook
On this page

Was this helpful?

How to synchronize data from Datastore to Google Spanner

In case of a large number of data to read and analyze (complex queries), it is possible to synchronize data initially stored in an entity of Google Datastore and duplicate it on a corresponding table on Google Spanner. Such a destination table must have exactly the same structure (field names and types) of the original one (entity on Datastore).

In order to do it:

  1. First, a Spanner instance must be configured . This will be used to host all Spanner tables to create and enquiry.

  2. The application parameter GOOGLE -> "Synchronize Datastore data with" must be filled in with the Spanner destination type. That means it is possible to synchronize data from Datastore entities only to this media.

  3. Use the "Duplicate Datastore object to BigQuery" command, available on the Data model list; this feature creates a new data model (having "spanner" type) starting from the selected Datastore type model; the new object has the same name and field names/types. If you need to align structural changes applied to the Datastore original object to the new one, you can add additional fields to the destination object.

  4. Use the "Sync data to db" checkbox in the data model definition window for the Datastore object: once checked it, every insert/update/delete operation applied to the Datastore entity will be duplicated in the target relational table; there is not a latency on the Datastore side: the duplication operation is always performed on a separated thread.

  5. Optionally, you can also refine the sync rule and limit data to duplicate, using a filtering condition (e.g. companyId==00000)

An alternative to the steps 4-5 is using the "Export from Datastore to Spanner" feature, where you can export the whole content (either as insert or update operations) and schedule this job to be execute on a regular basis, using a scheduled process.

Aligning data structure between Datastore (master repository) and Spanner

In the Spanner data model definition window, there is a "Align model from Datastore" button, which can be used to copy the Datastore entity definition (fields) to Spanner and all related objects connected to it (business components, panels). Use it to quickly align the two models.

It also ALTER Spanner table and add new fields coming from Datastore. This is helpful also when you export->import metadata from one environment to the other.

Important note: it would be better to opt for the "Sync data to db" approach, rather than the export, since it synchronize data in real time.

PreviousHow to synchronize data from Datastore to BigQueryNextHow to synchronize data from Datastore to CloudSQL

Last updated 2 years ago

Was this helpful?