Skip navigation
All Places > Developer > Blog > 2016 > May
2016


 

 

 

Have you registered for SugarCon yet? Remember that UnCon is right around the corner! We have 3 general sessions and 24 different technical breakout sessions at UnCon this year. That is over 24 hours worth of fantastic Sugar technical content that you do not want to miss!

 

Elasticsearch

 

We are doing multiple sessions on Elasticsearch which has been a very popular topic at past events.

 

Last year's UnCon provided an example customization that allowed full text search of file attachments on Documents and Notes modules. The Elasticsearch file search package is over in the UnCon Github repository, if you want to try it out for yourself. It is worth a look!

 

For this year, we want to do build something different. Do you know of an Elasticsearch use case that you would like to see implemented in Sugar? Have you been asked to do a geolocation search? Perhaps a request for some fuzzy analysis of search terms?Post a comment to the Elasticsearch Deep Dive abstract with your use cases. Our Elasticsearch wizards are standing by!

 

G11N, I18N, L10N and Accessibility

 

There are Sugar customer all over the world, so a big emphasis for our product is making sure it is ready for global users with a variety of languages, cultures, and accessibility challenges. The processes for preparing or designing software to support the needs of global and diverse user community has many names: globalization, internationalization, localization, and/or accessibility.

 

Have you struggled getting Right-to-Left support working in your Sugar customizations? Never used a screen reader before?Post a comment to the G11N, I18N, L10N, and Accessibility abstract with any of your web application challenges that we should address.

 

DevOps / Automation at Sugar

 

Do you have a DevOps process? Don't know where to get started? Allow our DevOps experts guide your way!Post a comment to the DevOps / Automation at Sugar abstract with the details of your development processes and technology that we can focus the discussion on solutions that matter to you.

 

And More!

 

There are dozens more sessions to talk about. Please, review the abstracts in the UnCon community to learn about them all!

 

We hope to see you in beautiful San Francisco.

 

 

 



To coincide with the release of Sugar 7.7, the Engineering team has released updated versions of Unit Tests and Performance tools.

 

Requesting access to Sugar Test Tools

 

Sugar Test Tools are in private Github repositories within the SugarCRM Github organization. Requesting to have your Github account added to the SugarCRM Github organization is easy, just fill out this form.  Visit the Developer Tools section of this site to learn more.

 

Sugar 7.7 Unit Tests

 

The Sugar 7.7 Unit Test framework is now available in the Sugar Unit Test repository.  This package includes all the test suites and unit test framework used to develop Sugar 7.7.

 

The Sugar 7.7 unit tests are in the 7_7_0 branch of the Unit Tests repository.

 

Sugar Load Testing Framework v2.5

 

The v2.5 release includes several new feature enhancements and support for load testing Sugar 7.7.  Some of the new features include the ability to adjust the rate that Create/Delete actions are run during a performance test as well as support for new APIs in Sugar 7.7 release.

 

The back-end load testing framework is in the backend-side branch of the Performance repository.

 

Sugar Client-Side Performance Testing framework

 

This is an all new release! The Performance team has created a framework for front-end (JavaScript) performance testing based on Cucumber.JS.  This framework can be used to automated your front-end performance tests to ensure that JavaScript performance and responsiveness does not degrade over time.

 

This client-side framework is in the client-side branch of the Performance repository.

 

Come to UnCon to learn more!

Learn how to use these tools from the Sugar Engineering team themselves at the UnCon at SugarCon on June 14th and 15th in San Francisco!

 

Join Yvan and Robert to learn more about writing Sugar unit tests and Alesia and Igor to learn more about Using Sugar Tools for Performance.

UPDATE July 24 2017

 

Sugar 7.9.0 introduced a Platform extension. This should be used for registering new platform identifiers for Sugar 7.9.0 and beyond.

 

What does the platform parameter mean in v10 REST API?

 

If you open your web browser's network tools, a login request to the v10 REST API used in Sugar 7.6 will typically look something like the example below.

POST /rest/v10/oauth2/token HTTP/1.1

 

Host: localhost

 

Cache-Control: no-cache

 

Content-Type: application/json

{

 

     "grant_type": "password",

 

     "username": "admin",

 

     "password": "admin",

 

     "client_id": "sugar",     "platform": "base",

 

     "client_secret": "",

 

     "current_language": "en_us",

 

     "client_info": {

 

         "current_language": "en_us"

 

     }

 

}

 

You can see in the request that we have specified a platform parameter called "base". This parameter is optional (the default is base) so even if you have used the v10 REST API before you may not have been aware of it or what it means.

 

Sugar Platforms

 

Platforms are used by Sugar to support the needs of multiple Sugar clients.  For each client, you are allowed to specify unique Sidecar Viewdefs, custom or modified platform API endpoints, as well as the ability to support a concurrent user session.

 

Sugar platform specific code is found under the clients/ and modules/MODULE_NAME/clients directories.  Sidecar client metadata is implemented under clients/*/layouts|views|fields and platform API endpoints are implemented under clients/*/api.  Sugar will fall back to base platform implementation when a more specific implementation is not available.

 

In general, the Sugar v10 REST API will only allow a single session (specifically, a single OAuth access token) to be created for any single platform at any one time.  For example, you cannot log into Sugar from multiple different web browsers on different computers at the same time.  This is necessary to prevent users from sharing subscriptions while allowing users to simultaneously access Sugar using a mobile device and their browser.

 

This is allowed by the fact that the SugarCRM Mobile app uses the mobile platform identifier while the web client uses the base platform identifier.

 

The number of sessions that is allowed per Sugar Platform is controlled by the SugarOAuth2StoragePlatform class that is associated with with a particular platform.

 

For example, if you look at SugarOAuth2StorageBase.php, you will see that the base platform allows only 1 standard web session at a time.

 

However, examining SugarOAuth2StorageMobile.php will show that we allow 2 mobile sessions at a time to allow access from multiple personal devices such as smartphones and tablets.

 

Thus it is possible to change the number of allowed concurrent sessions by overriding or implementing a new a SugarOAuth2StoragePlatform class for a custom platform.

 

Specifying new custom platforms will generate new copies of metadata cache

 

Today, Sugar supports the ability to specify arbitrary platforms.  This allows you the ability to create your own custom Sidecar based clients that are configured with their own custom metadata that extends from the base Sugar metadata.

 

A side effect though is that Sugar will build a metadata cache for each and every platform that it comes across.  Even if no custom platform metadata is specified, it will just inherit from base metadata.

 

Problems caused by random platform IDs

 

Some developers have followed our previous advice in order to allow an integration to maintain concurrent user sessions by creating their own unique platform IDs.  However, in some cases we have found integrations using random platform IDs which causes a lot of bad side effects.

 

For example, generating new copies of platform metadata takes time, so there can be an impact on user interface responsiveness.  Disk usage starts to swell and the size of an instance back up grows tremendously. Also, downstream integrations and systems can be affected due to slowed API responsiveness because of all this extra processing.

 

Sugar 7.6 Changes

 

We have added a new Sugar Configuration setting called disable_unknown_platforms that is currently disabled by default.  SugarCRM will selectively enable this setting on Sugar On-Demand instances that appear to be negatively affected by use of random platform IDs.

 

You can enable this setting on your own instance by setting up a config override.config_override.php

<?php

 

$sugar_config['disable_unknown_platforms'] = true;

 

When this setting is enabled then any custom platforms that are used must be registered via the custom directory.custom/clients/platforms.php

<?php

 

$platforms[] = 'base';

 

$platforms[] = 'mobile';

 

$platforms[] = 'portal';

 

// New one

 

$platforms[] = 'mine';

 

Changes in future Sugar versions

 

In the future, we plan to create an Extension for registering custom platforms. We also may enable this restriction more broadly in Sugar On-Demand. So it is important that Sugar Developers know how to properly use Sugar Platforms.

The Platform extension was added in Sugar 7.9.0.
We are gearing up big time for SugarCon so I wanted to give the community a quick update.

 

UnCon @ SugarCon 2016 Overview

Sugar Developers - UnCon is YOUR conference! Hosted by SugarCRM Engineering, UnCon is the developer conference within a conference. UnCon is your opportunity to learn, rub shoulders, and hack with the top developers and architects at SugarCRM. Whether you are a Sugar Developer ace trying to get answers to your deepest technical questions or a Sugar newbie trying to learn what the Sugar platform is all about, this where you can get the answers straight from the source.

 

This year it is again being held on June 14th and June 15th at the Hilton Union Square in San Francisco.To participate at UnCon, you must be registered for SugarCon.

 

UnCon Agenda

 

The UnCon agenda is now live in the UnCon space within our Sugar Community.  Abstracts for the sessions are linked from the agenda page, so you can get more details about what will be covered and who is presenting.

 

Check it out so you can plan your visit to SugarCon and make sure you do not miss a thing!

 

UnCon Hoodies and Beer are Back!

 

We will have a limited quantity of exclusive UnCon hoodies this year.  They will be for card carrying Sugar Developers from our community only!  Come to UnCon to learn how to make one of these your very own.

 



 

 

 

Also back by popular demand:

 

 

 



 

Give us Feedback!

 

Please provide feedback within the UnCon community!  You can respond with comments on abstracts, post any of your burning questions, and create new ideas for the UnCon event!

Post originally written by Emmanuel iNet.

 

Here is another guest post from Emmanuel Dyan from the Elite SugarCRM Partner iNET Process. In it he addresses a common Sugar project requirement using an open source tool developed by iNET Process.

 

The problem that we will try to solve with this post is:

How do we make sure that we are never developing using actual customer data but, at the same time, work with data that reflects reality?

 

Data Anonymization

 

Usually, when we work on a customization project for a customer you have a minimum of 3 different environments: Development,  User Acceptance Testing (UAT), and Production. To make sure that we work in optimal and consistent conditions, we usually copy the database from one environment to another (preferably from production to other environments). Doing this type of manipulation has multiple drawbacks, including:

  • We have to collect a Database Dump which means that it contains unencrypted raw data. What would happen if we mistakenly expose this dump to someone who is unauthorized?
  • We have to test some functionality to make sure that it works. What would happen if we test a Campaign that sends thousand of e-mails ... to the ... actual customers of our customer?

 

Anonymizing the data is the best practice to avoid "playing" with customer data and to keep their trust in us.

 

The challenge with anonymizing data is figuring out how to overwrite the data with something that is completely unrecognizable. For example: "John Doe" will become "voluptatem accusantium". His job title becomes "doloremque" and his country "magnam". His phone number will become "569898520114457878744778" instead of "+123456789".

 

Anonymization Tool

 

Now how do we work in realistic conditions with that kind of anonymization? Indeed, we need another solution that works with any Sugar instance. The solution we are demonstrating below is implemented in our open source CLI tool (sugarcli).

 

The anonymization architecture follows:

  • It uses another independent tool, called "neuralyzer" (by iNet Process) that is a command line tool to anonymize a database.
  • Neuralyzer uses a library called "Faker" to generate realistic data.
  • It is composed of two subcommands:
    • A configuration generator.  This is used to generate a configuration file automatically without destroying the system tables (config, relationships, etc).
    • The anonymizer that connects directly to the Sugar Database to perform the job. It uses iNET Process libsugarcrm to performs SQL queries (via a PDO connection). It also purges deleted records and cleans _cstm tables of deleted records. It finishes by then emptying all _audit tables.

 

Configuration Generator

 

The configuration generator reads all the Sugar database tables and tries to guess, from the field type or the field name what type of fake data needs to be generated. For example a field containing the string city will receive a random city.  If a field contains _user_id then it will be ignored in order to preserve unique IDs.

 

To use it, first download SugarCLI from https://github.com/inetprocess/sugarcli.

 

Then run:

./sugarcli.phar anonymize:config --path <sugarcrm_path>

 

The parameters are the following:

    --file=FILE                   Path to the configuration file [default: "../db/anonymization.yml"]    --ignore-table=IGNORE-TABLE   Table to ignore. Can be repeated (multiple values allowed)    --ignore-field=IGNORE-FIELD   Field to ignore. Can be repeated (multiple values allowed)-p, --path=PATH                   Path to SugarCRM installation.

 

Example

 

The command ./sugarcli.phar anonymize:config creates a file that looks like:

guesser_version: 1.0.0 entities:     accounts:         cols:             name: { method: company }             description: { method: sentence, params: [20] }             facebook: { method: url }             twitter: { method: url }             googleplus: { method: url }             account_type: { method: randomElement, params: [['', Analyst, Competitor, Customer, Integrator]] }             industry: { method: randomElement, params: [['', Apparel, Banking, Biotechnology, Chemicals]] }             annual_revenue: { method: randomNumber, params: [4] }             phone_fax: { method: phoneNumber }             billing_address_street: { method: streetAddress }             billing_address_city: { method: city }             billing_address_state: { method: state }             billing_address_postalcode: { method: postcode }             billing_address_country: { method: country }             rating: { method: sentence, params: [8] }             phone_office: { method: phoneNumber }             phone_alternate: { method: phoneNumber }             website: { method: url }

 

As you can see, the commands uses different methods to guess the type of faker methods to use:

  • If it is a dropdown then it gets the list of values from vardefs
  • If the field contains a known word it uses a pre-defined method (example .*_city = city)
  • Otherwise it will use the dbType (varchar = sentence)

 

You can change the content of the file once generated to match your criteria.

 

Run the anonymization

Warning! Do not run this on a production instance! This command will overwrite data in the target database. 
./sugarcli.phar anonymize:run --path <sugarcrm_path>

 

The parameters are the following:

    --file=FILE        Path to the configuration file [default: "../db/anonymization.yml"]    --force            Run the queries    --remove-deleted   Remove all records with deleted = 1. Won't be launched if --force is not set    --clean-cstm       Clean all records in _cstm that are not in the main table. Won't be launched if --force is not set    --sql              Display the SQL of UPDATE queries    --table=TABLE      Anonymize only that table (repeat for multiple values) (multiple values allowed)-p, --path=PATH        Path to SugarCRM installation.

 

 

 

Example

 

The command ./sugarcli.phar anonymize:run --table=accounts --force gives the following output:

Be careful, the anonymization is going to start That will overwrite every data in the Database !
If you are sure, please type "yes" in uppercase YES
Anonymizing accounts50/50 [============================] 100%
Emptying accounts_auditEmptying bugs_auditEmptying campaigns_auditEmptying cases_auditEmptying contacts_auditEmptying contracts_audit....
Done in 0.42 sec (consuming 40.5Mb)
Now you can dump your database and send it to your development team! If you use the parameters --remove-deleted and --clean-cstm, it will be smaller too.