Google App Engine
Checking for authorizations
GAE javascript actions are stateless web services, that is to say, they do not store conversational state among different calls.
These actions can be invoked directly or, even better, through an alias (api?cmd=...).
In any case, an authentication process is required, in order to successfully invoke an action.
Authentication can be performed in 2 alternative ways:
by invoking "login" standard ws first, get a token and use it for any subsequent invocation of a GAE action by pass the token along with the other input parameters
by invoking directly a GAE action an pass credentials along with the other input parameters
In any case, credentials are required and must be provided always as request headers.
Moreover, in the first case the needTmpToken is required too, in order to ask Platform for GAE to get back a token to use in the next requests.
Optionally, it is possible to ask Platform to fetch authorizations as well, when carrying out the authentication process. Authorizations are the list of granted roles associated to the specified user.
This can be carried out b y including in the request the "loadRoles=Y" header parameter.
Finally, the utils.checkRoleId(roleId) method can be included in any GAE action, in order to force Platform for GAE to check for that role as a role bound to the current user. This checking is performed ONLY IF "loadRoles=Y" has been previously included in the authentication request.
Reading data from Datastore
When there is the need to read a sigle entity, it is strongly recommended to use the following method to read a single record by key instead of executing a query where the filter condition limits the number of record, since reading a single record by key is cheaper.
Syntax
Parameters
Reading data from Datastore
When there is the need to read a sigle entity, it is strongly recommended to use the following method to read a single record by key instead of executing a query where the filter condition limits the number of record, since reading a single record by key is cheaper.
Syntax
Parameters
Execute a long HTTP(s) requestExecute a long HTTP(s) request
AppEngine is a high scalable web container, used to run web services.
In order to provide scalability, it limits all HTTP requests to 30 seconds only. It goes without saying that in case of longer operations, like in case of a sequence of writing operations, there is the need for a queue and the real operations must be postponed to the queue execution.
The best approach is to enqueue operations in a GAE action, using something like:
This method gets back a unique identifier for the element in queue.
At this point, there are two possible scenarios:
the client invoking the current action does not need a feedback: that's all
the client needs a feedback and this action must wait for the termination of the enqueued action
In the latter case, since the HTTP request launched by the client will terminate in 30 seconds (because GAE will force its termination), we can wait after the enqueueActionWithNoteAsString command, but not for more than 30 seconds. It can be helpful this method:
In this example, this method will wait up to 20 seconds, until all enqueued elements specified in list will be terminated.
If after the specified timeout (expressed in seconds), not all of the elements are still finished, the method ends gracefully and get back the list of not terminated element ids, which can be provided to the client.
Consequently, the client should invoked a second request, to a third action which should contain only the previous instructions, so that it can determine when the actions are terminated.
Capturing enqueued action outcome
The enqueued action can also provide a feedback to the invoking action or to any other action; the action return value (JSON content) is automatically cached internally, referred by the enqueued action uuid. That means that if the enqueued action terminates by providing some content (either a successful or a wrong outcome), like in this scenario:
this content can be accessed, for example from the calling action, through the uuid:
Bear in mind that cached response for an enqueued action is available up to 1 minute, after that time, the content is automatically removed from the cache.
Checking for element termination
When enqueuing an element in AppEngine, the "enqueueActionWithNoteAsString" method returns an uuid identifying the specific element appended to the queue.
Optionally, you can pass forward any payload (for example a unique identifier as String or a JSON String) to such method, using the "note" argument. You can later use this information to check for the element termination, using the "getElementFromQueueByNote" method, which enquiry the queue management system, searching from an element having the specified payload.
Syntax
The payload argument is used to search for an element still inside the queue, having that payload bounded.
If it exists, the method returns the element in queue (a JSON String representing the enqueued element).
An example of returned JSON is:
There are two possibilities for having still and element returned:
the element is still in queue; in such a case, the "status" attribute is set to "ENQUEUED"
the element has been dequeued but not processed correctly and error was fired during the elaboration; in such a case, the "status" attribute is set to "ERROR"
If the method returns null, it means the element has been already processed and removed from the queue.
Example
Thread where the element is enqueued:
Another thread, checking for element termination
Concurrent writing operations
As long as the updateObject/updateObjects are used, the Google Datastore automatically provides a checking which avoids the possibility to update the same object multiple times, starting from old states of the object: it means that if concurrent requests to writing operations are performed, only the ones starting from a consistent state from the Datastore will be allowed. The others will fire an exception, which must be captured and managed appropriately by the GAE javascript code.
Let's take the classical example of a counter table "Counters" whose purpose is to reckon the next value for a counter. A Datastore object used for that goal would be something like:
The right GAE javascript action should manage the possibility of a concurrent access exception and attempt multiple times to get the next value:
How to manage distributed transactions and SAGA
Since Google Datastore is a NoSQL database, the ACID (atomic, concurrent, isolated, durable) properties supported by a relational database cannot be applied.
Consequently, when multiple writing operations on different entity types are required, it is not possible to count on the Database behavior. The best way to manage a distributed transaction over multiple entity types, is to manage it as for a micro-service architecture, based on the SAGA pattern (see for example: https://blog.couchbase.com/saga-pattern-implement-business-transactions-using-microservices-part-2\, using the "command-orchestration sequencing logic", where a main javascript action would orchestrate all operations, by invoking each of them sequentially.
In case of an error fired by any of the invoked actions, the main javascript action has the duty to "undo" any writing operation already carried out, by deleting them. That means that every invoking object should not only provide a "writing operation" but also an "undo" operation, to clean up any writing operation already done.
When comparing Platform with the schema above:
the "Order Service" represents the GAE javascript action invoked by a web/mobile client
the "Order Saga Orchestrator" is the GAE javascript action enqueued by the previous one, which is invoked asynchronously
"Payment Service", "Stock Service" and "Delivery Service" represent additional GAE javascript actions which can be called by the previous enqueued action. In the easiest case, we can embed this logic as a sequence of writing operations within the "Order Saga Orchestrator" action, which has the purpose of managing the "rollback" operations as well, in case of errors during the sequence of writing operations. In the most complex scenario, these 3 additional actions are enqueued too by the "Order Saga Orchestrator" into:
3 distinct queues, if it makes sense to parallelize these 3 operations; a blockingWaitAllElements is required, in order to check out if all operations completed correctly or to manage the undo task
a unique queue, where the "Payment Service" is enqueued, waiting for its termination through the blockingWaitAllElements method; when completed correctly, the second actions is enqueued; same approach for the third. Undo operations are managed after each blockingWaitAllElements termination, if needed.
How to recycle enqueued elements with problems
Every time a GAE action enqueues an element to Google Cloud Task, the same element is also saved in the QueueElements entity by Platform. This entity contains:
id - element identifier
json - the input data to pass forward to the element to process
app id, company id, site id, namespace
status
The status attribute is set to ENQUEUED every time and element is enqueued.
When the Task manager extracts the element, it is processed and a few alternative outcomes can arise:
the elaboration terminates correctly: the record is removed automatically from QueueElements entity
the elaboration was interrupted by an error:
if the max number of retries has not reached yet, the record is marked with ERROR status and managed again, up to the max number of retries set
f the max number of retries has been reached, the record is marked with ERROR status: not more attempts and carried out and the record remains in QueueElements for ever
Another abnormal scenario is when an element is never extracted from the Task manager: in such a case, the record still remains in QueueElements with status ENQUEUED.
It is possible to use the following GAE javascript function to re-process a single element in QueueElements: after re-enqueuing it, the record is also removed from QueueElements.
It is possible to use the same GAE javascript function to re-process multiple elements in QueueElements: the ones matching the filtering conditions passed forward to the method; after re-enqueuing a record matching the filtering criteria, that record is also removed from QueueElements.
Execute an HTTP(s) connection using NLTM authentication
result expressed as a String (e.g. a JSON or XML result content)
Syntax
Details
HTTP response codes included between 200 and 399 are managed as correct answers and the response is sent back through the "json" return variable.
In case of HTTP response codes above or equal to 400, an exception is fired an the exception content would contain the message sent back by the invoked web service; consequently, it would be better to surround this instruction between try-catch.
The "settings" js object can include the following attributes:
"connectionTimeout: an optional number defining the timeout for the connection
"headers": a js object containing request headers, e.g. required credentials
Example:
Call a business component from a server-side js business component
It is available a js function to invoke from within a server-side js b.c. (not from an action) to call another business component.
Basically, it can be used to pass forward all request parameters received in input from the starting b.c and get a response to use as the final result to pass back.
Syntax
Example of a server-side js b.c. invoking a b.c. for grids
Execute a long HTTP(s) requestExecute a long HTTP(s) request
AppEngine is a high scalable web container, used to run web services.
In order to provide scalability, it limits all HTTP requests to 30 seconds only. It goes without saying that in case of longer operations, like in case of a sequence of writing operations, there is the need for a queue and the real operations must be postponed to the queue execution.
The best approach is to enqueue operations in a GAE action, using something like:
This method gets back a unique identifier for the element in queue.
At this point, there are two possible scenarios:
the client invoking the current action does not need a feedback: that's all
the client needs a feedback and this action must wait for the termination of the enqueued action
In the latter case, since the HTTP request launched by the client will terminate in 30 seconds (because GAE will force its termination), we can wait after the enqueueActionWithNoteAsString command, but not for more than 30 seconds. It can be helpful this method:
In this example, this method will wait up to 20 seconds, until all enqueued elements specified in list will be terminated.
If after the specified timeout (expressed in seconds), not all of the elements are still finished, the method ends gracefully and get back the list of not terminated element ids, which can be provided to the client.
Consequently, the client should invoked a second request, to a third action which should contain only the previous instructions, so that it can determine when the actions are terminated.
Get directory path
Syntax:
var path = utils.getDirectoryPath(directoryId)
Caching values
Available methods:
addValueInCache(String varName,Object value) - expiration in 1 day
addValueInCache(String varName,Object value,Long expirationTime)- expiration expressed in minutes
removeValueInCache(String varName)
removeValuesFromCache(String keyPrefix) - Remove multiple values from cache: the one whose key starts with the specified pattern.
invalidateAll()
Object getValueInCache(String varName) - return if the variable is stored in cache, it returns the corresponding value, otherwise an exception will be fired
var string = clearCache(String varNamesStr,String keysStr) - Invoke GAE instance and ask for clearing up a list of variable names from the MemCache. Alternatively, if keys is specified as well, it removes only part of it. Arguments:
varNames a varName or many variable names separated by a comma (e.g. "MOTPROM,MOTPROM_BANCHES")
keys optional: can be null; a key within or many keys, separated by a comma; if specified, the cached entry related to varName must be a map and such a map inside the cache will not be removed completely, but only the entries related to the specified keys
SQL query execution with very long result set
In case of a SQL query returning a very long result set, it is NOT recommended to use the previous method, since it gives back the whole result set at once, which is dangerous because it can consume a large amount of memory on the server.
A better solution is represented by the following method, where the result set is read row by row: for each row, a callback function is invoked, in order to process it.
Consequently, this method cannot be coupled to the user interface, since it would mean that all data would be read in the end. This approach is good when the row processing does not involve the UI, but some other operation on the server side, like writing a text file, starting from the result set.
Syntax
Details
Generic SQL execution (NO SQL queries) without logging it
Syntax
Details
Note: the SQL operation will not be logged. This method csan be useful with bulk operations, whose execution could slow down if a log message were produced for each execution.
Generic SQL execution (NO SQL queries)
Syntax
Details
Execute a GQL query on Google Datastore
The datastore must be already configured as a global parameter. Once done that, it is possible to execute a query statement, in order to fetch a list of entities. The query language is GQL: filtering and sorting conditions are strictly ruled by the Google datastore. That means that additional indexes could be defined before executing the query. For instance, = operators can be used without additional indexes, but it is not so for sorting conditions or filtering conditions having not equal operators (e.g. <, <=, etc.). See Datastore syntax to get detail information about the syntax to use when filtering entities.
Syntax
Details
Example
Note: every GQL instruction will be logged.
Note: in case of a data model where there are attributes having type Array, this method will get back also the array value, expressed as a String whose values are separated by a comma.
Important note: please use the cached version of this method as often as possible:
where maxCachedEntities is the max number of cached entities having the same entity name specified in the GQL query.
Insert a single entity into the Google Datastore
The entity is expressed as a Javascript object.
The datastore must be already configured as a global parameter.Once done that, it is possible to execute operations on the Google Datastore.
Syntax
Details
Note: in case of a data model where there are attributes having type Array, this method will get back also the array value, expressed as a String whose values are separated by a comma.
Update a single entity into the Google Datastore
The entity is expressed as a Javascript object
The datastore must be already configured as a global parameter.Once done that, it is possible to execute operations on the Google Datastore.
Syntax
Details
Note: in case of a data model where there are attributes having type Array, this method will get back also the array value, expressed as a String whose values are separated by a comma.
Delete a single entity from the Google Datastore
The entity is expressed as a Javascript object.The datastore must be already configured as a global parameter.Once done that, it is possible to execute operations on the Google Datastore.
Syntax
var json = utils.deleteObjectOnGoogleDatastore(obj, dataModelId, interruptExecution);
Details
ArgumentDescriptionobja Javascript object related to the entity stored in the Datastore and to removedataModelIdit identifies the data model having "datastore" type, related to the entity to removeinterruptExecutionboolean flag used to define if the executing of the current server-side javascript program must be interrupted in case of an error during the execution of the operationoktrue in case of the operation has been executed successfully, an exception otherwise
Get a parameter value
Syntax
var paramValue = utils.getParameter(paramName)
Details
ArgumentDescriptionparamNamea string value representing a parameter named defined at installation level (CON44 table) or at application level (CON07 table)
paramValue the value defined for the specified parameter name or null if not found; if the parameter is found as an application parameter, that value is returned, otherwise it will be returned the value defined at installation level
Get a custom user parameter value
Syntax
var value = utils.getCustomApplUserVars(String varName)
Replace internal representation of a javascript String (CompString) with a String
Syntax
var vo = utils.fixVo(vo)
Execute a javascript GAE action starting from another one
Syntax
var json = utils.executeAction(actionId, vo, params, headers);
Last updated