Skip navigation
All Places > Developer > Blog > Authors Matt Marum
1 2 3 Previous Next

Developer

104 Posts authored by: Matt Marum Employee

Hello Sugar Developers!

 

We want to make sure your customizations and integrations are ready for the Sugar Winter ‘19 release, so we’re hosting two webinars just for you!

 

Webinar Topics

 

We’ll discuss the big changes that are likely to impact you including the following:

 

  • New REST API version: v11_4
  • New Comments Log field type and APIs
  • Enhancements to Reports module
  • Upgraded PHP and JavaScript libraries including jQuery

 

Webinar Schedule

 

We are holding two sessions to accommodate various geographical locations. Please choose ONE of the following times below.

 

Thursday, December 13th 7:00 - 8:00 AM PT

OR

Thursday, December 13th 5:00 - 6:00 PM PT

 

Register Now!

 

As always, we will be posting the webinar recordings to the Developer community  for those who are unable to attend the live sessions.

We are fast approaching the Winter '19 release into the Sugar cloud that is planned for January 2019. Some of the changes coming in this release will be upgrades to the jQuery library and associated plug-ins that are commonly used by user interface customizations to Sugar.

 

jQuery libraries upgraded from v1.12.4 to v3.3.1

 

In the Sugar Winter '19 release, we have made a significant upgrade to the jQuery versions used by Sidecar and SugarMVC (BWC) modules. 

jquery: v1.12.4 -> v3.3.1 (changelog) (Upgrade Guide)
jquery-migrate: v1.4.1 -> v3.0.1 (changelog)

 

jQuery Migrate should help maintain compatibility for custom code using older jQuery APIs. Custom code that uses deprecated jQuery APIs should be updated as soon as possible.

 

As part of the jQuery upgrade, the following jQuery plug-ins needed to be updated to be compatible with v3.3.1.


include/javascript/jquery/markitup: v1.1 -> v1.1.15 (changelog)
include/javascript/pmse/lib/jquery.layout-latest.js: v1.3.0 -> v1.6.3 (changelog)
include/javascript/jquery/jquery.timepicker.js: 1.8.8 -> 1.11.14 (changelog)

 

If you've added jQuery plug-ins to Sugar as part of your customizations, you should make sure they are compatible with jQuery 3.3.1 as well! Check out the jQuery Plugin Registry to make sure the version of the plug-in you are using is compatible with jQuery 3.3.1. If not, you may need to upgrade to a newer version of your plug-in or remove it.

 

 

This plug-in is compatible with jQuery 3.3.1.  SWEET!

 

 

We've also upgraded the Bootstrap JS library from v2.0.0 to v2.2.1 (changelog) and also moved it to a new location.

 

include/javascript/jquery/bootstrap/bootstrap.min.js (v2.0.0) has been removed
include/javascript/twitterbootstrap/bootstrap.min.js (v2.2.1) has been added 

 

If you happen to refer to the old file directly, you'll need to update your links to point to the the new location.

 

Other JavaScript library updates

While jQuery is probably the most impactful JavaScript change in this release, there are other JS libraries that you should know about.

 

Sidecar library updates

We've also made some changes to these other two Sidecar libraries. If you happen to be using them, please check out their change logs to ensure that you're still compatible.

 

sugarcrm/ventana: v1.1.1 -> v1.1.2 (changelog)
moment: v2.11.2 -> v2.22.2 (changelog)

 

Removal of JIT Chart library

The deprecated JIT chart library has been removed in this Sugar release. The JIT chart library was deprecated in Sugar 7.10 and has been replaced by Sucrose charts. Please ensure you have removed any code that uses JIT charts prior to upgrading.

 

Don't miss the Winter '19 Sugar Developer Webinar!

On Thursday, December 13, come join us in one of our live webinars about all the enhancements coming in Winter '19 to the Sugar cloud.

We can probably all agree that automated tests are great, but sometimes we put off writing them because the initial first step of getting everything working correctly can be really difficult.  

 

Good news, Sugar developers!  Writing PHPUnit tests for your customizations just got easier in Sugar Fall '18!  Our engineering team has created a test runner specifically for your custom PHPUnit tests.

 

If you do not have access to the SugarCRM provided unit test suite, submit the Developer Tools Request.

 

The first thing you'll need to do is add the SugarCRM provided unit test suite to your Sugar directory. Be sure to select the branch in the unit test suite that matches the version of your Sugar instance (needs to be 8.2 or higher). Follow the instructions in the branch's readme to add the tests to your Sugar directory, install the necessary dependencies, and update the file permissions.  

 

Now you're ready to write your tests!  Add your new PHPUnit test classes to the custom/tests/unit-php directory. You may need to create it yourself if it doesn't already exist.

 

For example, create a new file named CustomLogicHookTest.php and place it in the custom/tests/unit-php directory.  Paste the following in to the new file:

<?php
    //custom/tests/unit-php/CustomLogicHookTest.php
   
    namespace Sugarcrm\SugarcrmTestsUnit\custom;
   
    use PHPUnit\Framework\TestCase;
   
    /**
     * @coversDefaultClass \CustomLogicHook
     */

    class CustomLogicHookTest extends TestCase
    {
       
        /**
         * @covers ::aFunctionInsideTheCustomLogicHook
         */

        public function testCustomHook()
        {
            // Wow! This is a horrible test.
            // You would never do something like this just to increase your test  
            // coverage, right?
            $this->assertTrue(true);
        }
    }

In a shell, navigate to the Sugar provided unit tests directory by doing something like the following:

cd tests/unit-php

Then you can run the tests by executing

php ../../vendor/bin/phpunit --testsuite custom

or by using gulp (gulp installation instructions are available here)

gulp test:unit:php --suite custom

The above test is extremely simple and doesn't test anything. If you'd like to see an example of a larger test suite that tests Sugar customizations, check out Professor M's School for Gifted Coders.  We've recently updated our PHPUnit test suite so we can use the new test runner.  Check out the pull request associated with this change for the details.

This blog by SugarCRM Technical Account Manager and Solution Architect Angel Magana is reposted with his permission. See Angel's Blog: SugarCRM: Related Data Interactions for the original post from Angel's blog. In this post, Angel explains some best practices for working with related records in PHP that can help improve performance of your customization.

Fetching related records

 

Often times a customization we are working on requires us to interact with related data, usually records that have a parent-child relationship with one another. For example, a customization might require us to determine if a given Account record has 1 or more related Contacts. Or for that matter, we may need the collection of ID values that represent all the Calls linked to a Contact.

 

In either example, the path most often followed could be described with the following pseudo-code:

 

1. Instantiate the parent SugarBean object (e.g. Account/Contact record object)
2. Load the corresponding relationship to access the related data objects
3. Retrieve the related SugarBean objects representing the related objects

 

See below for a PHP snippet illustrating the above:

 

<?php 

//Some other code

//Instantiate parent record SugarBean object
$ParentBean = BeanFactory::retrieveBean('Accounts', 'some_account_id_value');

//Load the pertinent relationship using Link field name
$ParentBean->load_relationship('contacts');

//Retrieve related Contact(s) SugarBean object(s)
$RelatedContacts = $ParentBean->contacts->getBeans();

//The rest of our code


Line 12 in the above example would effectively create an array of SugarBean objects representing the linked Contacts. From that point forward, determining whether or not the Account has any linked Contacts becomes a simple matter of checking the size of the array. 

 

For the linked Calls example, the code would be very similar, except we would load the 'calls' relationship and the ID values we want would be in the SugarBean object array that results from executing the getBeans() method.

 

Do you need full Beans or just the IDs?

 

Now, all of the above would function just fine, but let us consider a few things about this approach.

 

First, upon executing the getBeans() method, we are asking Sugar to retrieve the complete SugarBean object for all the related records of a given type. However, for both examples, simply retrieving the ID value would have sufficed.

 

That is, for our purposes, a list of ID values in the $RelatedContacts array is just as good as an array of all the SugarBean objects. That array would still allow us to properly answer the question of whether there are or are not any linked contacts, solely based on the number of ID value entries the array contains.

 

In the case of the second example, we specifically only need the ID values of the related Calls, but getBeans() gives us everything pertaining to each of those calls. This tells us there is a lot of overhead we can trim.

 

From a performance standpoint, anytime we can optimize our code to retrieve only what we need, we help minimize wait times a user experiences. 

 

How do we then change our code to only give us the ID values?

 

It is actually a quite simple.

 

<?php 

//Some other code

//Instantiate parent record SugarBean object
$ParentBean = BeanFactory::retrieveBean('Accounts', 'some_account_id_value');

//Load the pertinent relationship using Link field name
$ParentBean->load_relationship('contacts');

//Retrieve related Contact(s) IDs ONLY.
$RelatedContacts = $ParentBean->contacts->get();

//The rest of our code

 

Examining the Link2 class in Sugar reveals a get() method which specifically only returns ID values of related data. Using get() in place of getBeans() would allow us to achieve the goal described in our examples and also reduce the performance overhead.

 

Side note: A similar method exists in the TeamSet class, named getTeamIds(), to retrieve the ID values of Teams that compose a TeamSet, without needing to retrieve all of the Team SugarBean objects. Also be aware that get() will return all records, irrespective of team assignments on the related records. 

Hello Sugar Developers!

 

We want to make sure your customizations and integrations are ready for the upcoming Sugar Summer ‘18 release, so we’re hosting two webinars just for you! The Summer '18 release is just a couple short weeks away.

 

What we will be covering:

We’ll discuss the big changes that are likely to impact you including the following:

  • Changes to Login screen and authentication classes
  • Emoji support
  • Built-in support for Double Opt-In (DOI)
  • Improvements to Reports and Advanced Workflow
  • SugarCon Update

 

Webinar Information:

We are holding 2 sessions to accommodate various geographical locations. Please choose ONE of the following times below.


Monday, July 2nd, 5:00 - 6:00 PM PT 
OR 
Tuesday, July 3rd 8:00 - 9:00 AM PT 
(Choose one)

 

Register Here

 

We will be posting the webinar recordings to Developer community for those who are unable to attend the live sessions.

 

P.S. Did you know you can apply for free passes to SugarCon by speaking at the conference or becoming a Sugar Scholar? Get the details on Two fabulous ways to earn a FREE pass to SugarCon 2018.

We have hit a tipping point in world wide Emoji usage. We really must work like we live, because the perfect two character combination of pizza and beer and others like it are starting to appear in business emails and other business communications. In particular, marketers have started to use emojis in subject lines of emails as a way to catch people's attention.

 

As Rich Green put it:

It's nearing the end of the second decade of the third millennium.

If people want to use poop emoji's in business, so be it.

 

Many Sugar customers have started running into an issue with emojis when using MySQL databases or the Sugar cloud. What could be so challenging about a poop emoji, you may ask? For that you may need a quick lesson on Unicode.

 

Character encoding and the beginnings of Unicode

Many of you are programmers, so you likely know that computers run entirely on (binary) numbers. As you type away on your keyboard, the stream of characters that gets stored in memory is a sequence of numbers based on the character encoding in use. "A" could be stored as 65, while "a" could be stored as 97, and so on. Programmers are obsessed with efficiency, so they would create many different character encodings for different alphabets or character sets to only use as little space as possible.

 

This became a real challenge as we started to share documents between systems via e-mail or the world wide web. Identifying and using the right character encoding became very important and also very difficult. If you didn't know the character encoding then you had to guess and if you guessed wrong then the text would not be displayed properly at all.

 

Encoding gone wrong. By Wikimedia Foundation (Wikimedia Foundation) [<a href="https://creativecommons.org/licenses/by-sa/3.0">CC BY-SA 3.0</a> or <a href="http://www.gnu.org/copyleft/fdl.html">GFDL</a>], <a href="https://commons.wikimedia.org/wiki/File:Mojibakevector.png">via Wikimedia Commons</a>

 

Fortunately, a group of companies, organizations, and individuals decided to come together to form the Unicode consortium. The Unicode consortium's primary mission has been to standardize character encoding into the Unicode standard that is widely used today. The most common Unicode character encoding is UTF-8 which is a variable length character encoding that can store any Unicode code point (which includes a growing number of emojis) into a number that is 1 to 4 bytes long.

 

MySQL's utf8 and utf8mb4 encodings

Programmers are obsessed with efficiency. Many programmers will not use 4 bytes if they think 3 bytes will do. When MySQL first implemented UTF-8 years ago, they decided that supporting the Basic Multilingual Plane (BMP) was good enough since it contained all the characters and most symbols used in modern languages. These characters require no more than 3 bytes. Adding support for the Supplementary Multilingual Plane (SMP) which includes emojis would mean that some characters would need to be stored in 4 bytes, which was just one byte too many for somebody.

 

Whatever the reason, MySQL's first implementation of UTF-8 that they called utf8 did not fully support UTF-8. When they fully implemented UTF-8 years later, they called it utf8mb4. This has been a source of confusion for many years since MySQL's utf8 character set seems to work fine until you run into certain special characters like emojis. Trying to store an emoji in MySQL's utf8 results in failure and the loss of data.

 

Sugar Summer '18 migrates from utf8 to utf8mb4

Sugar Summer '18 will use utf8mb4 for MySQL. This will allow users to import and display records and emails that contain emoji and other characters in the SMP. This feature will also be rolled into a future on-premise release.

 

The Sugar Summer '18 release will be available for Sugar cloud customers in a few short weeks.

 

Emojis in Sugar!

 

This character set and collation will be set automatically for new instances of Sugar and updated during the upgrade of existing instances of Sugar.  

 

If you have changed the default DB collation and you are in the Sugar cloud or on-premise using a MySQL database, then you should ensure that your collation is utf8mb4 compatible prior to upgrade. The collation can be set by the admin in Admin > Locale > Collation or by modifying config_override.php. If your collation is utf8mb4 compatible, the upgrade will automatically migrate the collation to utf8mb4. For example, if you have set your collation to be utf8_swedish_ci, the upgrade will migrate the collation to utf8mb4_swedish_ci. If no collation is set, Sugar will use the default utf8mb4_general_ci.

 

Database tables with very large row sizes (for example, custom tables with a large number of custom fields) may be unable to be automatically upgraded. The upgrader will notify you if a table would exceed the single-row size supported by MySQL (65,535 bytes) upon conversion to utf8mb4. In order to reduce the row size, we recommend the following:

  • Remove any/all fields that are not being used.
  • Reduce the lengths for char/varchar fields (e.g. size of longest existing value plus some padding).
  • Replace large varchar fields with text fields. Text fields are roughly 10 bytes, so significant size reduction exists when text fields can replace large varchar fields (e.g. VARCHAR(255).

For years SugarCRM has published database schema information for each new Sugar release at apidocs.sugarcrm.com. Generated documentation, like our database schema reference, has largely taken a back seat to more formal documentation like the Sugar Developer Guide and Sugar Developer training curriculum over at Sugar University.

 

Over the past couple of releases, we have improved the Sidecar framework code to have better structure and code documentation while preserving compatibility. One of the benefits has been a significant improvement in the quality of our generated Sidecar JavaScript documentation when using tools like JSDoc.

 

Anyone can now view the documentation at anytime for each new Sugar release without having to manually generate the documentation!

 

If you visit SugarCRM Generated Documentation, you will find the Sugar 8.0.0 Sidecar framework documentation available today. We will be publishing the generated Sidecar framework documentation for each new release in the future.

 

We plan to continue to invest in the quality and quantity of generated API references that we publish for each release.

Sugar Summer '18 introduced confirmation links to allow customers to gather affirmative consent for using email addresses in campaigns. Check out Sugar Summer '18 (8.1.0) Overview for Sugar Developers for more details.

 

As discussed in Why SugarCRM developers should care about Data Privacy and GDPR, data privacy is an important topic that we are going to keep discussing here on the developer blog. Today is another installment specifically focused on best practices for getting consent to collect and process personal information.

 

General Data Protection Regulation (GDPR) has strict requirements for consent. However, consent to personal data processing is a big topic for regulation in many countries and especially applies to email marketing. Under GDPR, unless you have an existing lawful basis for using personal information, you need to collect explicit consent from the data subject when storing and processing their information in Sugar. By explicit consent, we mean it was freely given, informed, affirmative, and unambiguous. For example, a web to lead form with a pre-selected opt-in checkbox is not going to cut it in the EU, Australia, or Canada.

 

Let's look at a couple different ways of collecting explicit consent.

 

Collecting explicit consent

 

Collecting explicit consent is particularly important for marketing campaigns and business development. For other CRM use cases (like customer support), you may already have another lawful basis that gives you the right to process customer information.

 

Consent really just boils down to allowing someone to knowingly and freely give their permission for you to use their personal information. You could collect consent over the phone, written in an email, or over a handshake in person.

 

It is possible to collect explicit consent with a properly designed web form as well. There are lots of resources out on the web with advice for building compliant Single Opt-In (SOI) web forms.

 

However, a common practice for collecting consent is to use Double Opt-In (DOI) which is also known as Confirmed Opt-In (COI).

 

Double Opt-In (aka Confirmed Opt-In)

 

We all know web forms aren’t perfect. Sometimes users aren’t sure what they are signing up for themselves. Maybe they mistyped their email address and now somebody else is signed up for that mailing list. Or worse, somebody is a troll and intentionally signing them up for email or services that they don’t want. There is a better way and it is called double opt-in.

 

Double opt-in (or Confirmed Opt-In) essentially means that someone who puts their personal information into a web form must confirm twice. A user enters their information on a form and affirms that they want to subscribe to a service. They then get a confirmation email, text message, or some other communication where they can re-confirm their intention. This is typically done by providing a link in an email that users must then click to confirm.

 

Double opt-in is an easy way to ensure you have a clean mailing list or lead list full of people who went through the trouble to confirm twice.

 

Many of our customers use marketing tools such Act-On, Marketo, and others to manage their mailing lists and the necessary opt-in processes. As an added feature to Sugar’s Marketing capabilities, we are planning to include DOI/COI in a subsequent release. Meanwhile, here is how you can achieve DOI/COI with Sugar 8 or Spring ‘18.

 

Adding Double Opt-In to Sugar 8.0.0

 

If you have access to the Sugar 8 Preview then this is something you can try today. Otherwise you will need to wait until the official release in a couple short weeks.

 

This customization implements double opt-in behavior for specific email addresses stored within Sugar. The steps below involve Sugar Studio configuration changes as well as code level customizations. This example allows for any email address to be subject of a confirmed opt-in flow without any specific dependencies on Campaigns, web to lead forms, Leads, Contacts, etc. Note that since this customization is focused on the confirmation step of double opt-in so for this reason we prefer the term “Confirmed Opt-In”.

 

Once you have followed the steps below, you will be able to generate confirmation messages for new email addresses. For each email address that needs to be confirmed, there will be an associated URL that, when clicked, will confirm and opt-in the associated email address.

 

Step 1) Configure your lead forms or integrations to opt-out by default

 

You need to take some steps to make sure that email addresses are not prematurely opted-in within Sugar.

 

In Sugar administration, visit the “System Email Settings” panel and make sure “Opt-out new email addresses by default” is checked. You may need to enter SMTP Mail Server settings in order to save this change on this page. This is a new setting in Sugar 8.0.0.

 

 

If you are using Sugar web-to-lead forms, they should be updated to remove the email opt-in checkbox that Sugar includes by default if it still exists. I’d recommend updating the form to let the user know that a confirmation email will be sent separately.

 

 

If you are using some other system (like a lead portal or marketing automation tool) to capture contact information, they should be configured to not send back email opt-out information to Sugar. Or alternatively they could be configured to set email opt-out to true by default.

 

Step 2) Add Studio configuration changes to Data Privacy module

 

In this example, we will reuse the Data Privacy module (new in Sugar 8.0.0) to track email confirmation activities. If you haven’t already, you may need to add the Data Privacy module to the list of displayed modules in the Administration panel first.

 

Add a new “Confirmed Opt In” option to the dataprivacy_type_dom used by the Data Privacy module’s type field. This will be used to differentiate email confirmations from other Data Privacy activities.

 

 

Also add two new custom fields that are visible only for Confirmed Opt-In activities. To do this, I used the following visibility formula on these custom fields.

 

equal($type, "Confirmed Opt In")

 

The first is an email_address_to_confirm_c text field that stores the record ID for the email address that needs to be confirmed.

 

 

The second is a URL field called confirmation_url_c that will use a generated URL. The default value is designed to pass email_address_to_confirm_c field and the Data Privacy id field as parameters to a custom entry point that we will create later. The URL will vary based upon the site URL for your Sugar instance. For my development environment, I used:

http://localhost:8080/sugar/index.php?entryPoint=confirmEmail&email={email_address_to_confirm_c}&id={id}

 

 

Make sure to add these two new fields to your Data Privacy record layout as well which will help us with testing. Remember to Save and Deploy after adding them.

 

 

Step 3) Add the ConfirmedOptInManager class and related Entry Point

 

The following class includes two optional logic hooks as well as a required method that will be used by a new Entry Point. We are choosing to use a custom Entry Point here instead of a REST API endpoint because the Sugar REST API is designed to only return JSON responses. This makes for a poor usability experience for the person who is trying to confirm their email address and sees a page full of gobbledygook in their browser. Typically, people will expect to see a nicely formatted web page with a confirmation message. An entry point allows us to return an HTML response that matches user expectations.

 

The createConfirmedOptIn() method can be used as an after_save hook on the Leads module to automatically create a confirmed opt-in activity whenever new Lead records are created.

 

The sendConfirmOptInMessage() method is a stub that can be used as an after_save hook on the Data Privacy module to automatically send confirmation emails to end users. This could also be orchestrated using an Advanced Workflow.

 

The last confirm() method is the most important one. This method is called directly by a custom public entry point shown further below. This is what determines if the request is a valid one and then updates the email address record. It first loads the DataPrivacy bean, then compares the email address ID stored there with the one provided in the URL. It then updates the EmailAddress and DataPrivacy beans accordingly.

 

./custom/Extension/modules/Leads/Ext/LogicHooks/createConfirmOptInRecord.php

 

<?php
// Copyright 2018 SugarCRM Inc.  Licensed by SugarCRM under the Apache 2.0 license.
$hook_array['after_save'][] = array(
  1,
  'create_confirmation_record',
  null,
'Sugarcrm\\Sugarcrm\\custom\\ConfirmedOptIn\\ConfirmedOptInManager',
  'createConfirmOptInRecord',
);

 

./custom/Extension/modules/DataPrivacy/Ext/LogicHooks/sendConfirmOptInMessage.php

 

<?php
// Copyright 2018 SugarCRM Inc.  Licensed by SugarCRM under the Apache 2.0 license.
$hook_array['after_save'][] = array(
  1,
  'send_confirmation_message',
  null,
'Sugarcrm\\Sugarcrm\\custom\\ConfirmedOptIn\\ConfirmedOptInManager',
  'sendConfirmOptInMessage',
);

 

./custom/src/ConfirmedOptIn/ConfirmedOptInManager.php

<?php
// Copyright 2018 SugarCRM Inc.  Licensed by SugarCRM under the Apache 2.0 license.
namespace Sugarcrm\Sugarcrm\custom\ConfirmedOptIn;

/**
*
* Manages confirmed opt-in workflow for Email addresses
*
* Class ConfirmedOptInManager
*/

class ConfirmedOptInManager
{

    /**
     * After Save logic hook on Leads module (or any other Person module with an email address)
     *
     * Used to automatically create email confirmation records in DataPrivacy module.
     *
     * @param \SugarBean $bean SugarBean with an email address that needs to be confirmed.
     * @param string $event Logic hook event name. Expected to be 'after_save'.
     * @param array $arguments
     */

    public function createConfirmOptInRecord(\SugarBean $bean, string $event, array $arguments)
    {
        //If this is a newly created record
        if (!$arguments['isUpdate'] && $bean->load_relationship('emails')) {
            $address = $bean->emailAddress->getPrimaryAddress($bean);
            //And a primary email address is set
            if (!empty($address)) {
                //Create a new 'Confirmed Opt In' DataPrivacy activity
                $dpb = \BeanFactory::newBean('DataPrivacy');
                $dpb->name = "Confirm $address";
                //Note: This assigned user for Data Privacy activity MUST have the Data Privacy Manager role!
                $dpb->assigned_user_id = $bean->assigned_user_id;
                //We will store ID for primary address here.
                $dpb->email_address_to_confirm_c = (new \SugarEmailAddress())->getGuid($address);
                $dpb->type = 'Confirmed Opt In';
                $dpb->save();
            }
        }
    }


    /**
     *
     * After save logic hook on Data Privacy module
     *
     * Can be modified to send the confirmation message to the appropriate email address.
     * You should make sure not to send multiple messages for the same Data Privacy request.
     *
     * @param \DataPrivacy $bean DataPrivacy bean that has been saved
     * @param string $event Logic hook event name. Expected to be 'after_save'.
     * @param array $arguments
     */

    public function sendConfirmOptInMessage(\DataPrivacy $bean, string $event, array $arguments)
    {
        //Should only continue if this is a newly created Confirmed Opt In request
        if ($bean->type === 'Confirmed Opt In' && !$arguments['isUpdate']) {
            $log = \LoggerManager::getLogger();
            $eab = \BeanFactory::retrieveBean('EmailAddresses', $bean->email_address_to_confirm_c);
            if (!empty($bean->email_address_to_confirm_c) && !empty($eab->email_address)) {
                $log->info(
                    "Send out confirmation to $eab->email_address with link to $bean->confirmation_url_c."
                );
                // You could update to send message using mailer, SMS gateway, whatever.
            }
        }
    }

    /**
     *
     * Opt-in the email address if the email associated with given requestID matches the given emailAddressID
     *
     * Used with `confirmEmail` public custom entry point
     *
     * @param string $requestId
     * @param string $emailAddressId
     * @return bool
     */

    public function confirm(string $requestId, string $emailAddressId): bool
    {
        global $current_user, $timedate;
        // Since this is a public endpoint, we need to select a current user to associate with this action.
        // For our purposes, we will use the User assigned to the Data Privacy request since the person confirming
        // their email address is likely not a current Sugar user.
        // Note: For this to work, assigned/current user must be admin or have Data Privacy Manager role!
        if ($current_user && empty($current_user->id)) {
            $q = new \SugarQuery();
            $q->from(\BeanFactory::newBean('DataPrivacy'), array('team_security' => false));
            $q->where()->equals('id', $requestId);
            $q->select(array('assigned_user_id'));
            $results = $q->execute();
            //If no assigned user, then default Admin user will be used.
            $assignedUserId = empty($results) ? 1 : $results[0]['assigned_user_id'];
            $current_user = \BeanFactory::retrieveBean('Users', $assignedUserId);
        }

        $confirmRequest = \BeanFactory::retrieveBean('DataPrivacy', $requestId);
        //Make sure email address matches the one associated with the request ID
        if ($confirmRequest && $confirmRequest->email_address_to_confirm_c === $emailAddressId) {
            //If already closed, then no need to re-confirm.
            if ($confirmRequest->status === 'Closed') {
                return true;
            }
            $addrBean = \BeanFactory::retrieveBean('EmailAddresses', $emailAddressId);
            $addrBean->opt_out = false; //opt in!
            $addrBean->save();
            $confirmRequest->status = 'Closed';
            $currentTime = $timedate->getNow();
            $confirmRequest->resolution = 'Confirmation received on ' . $currentTime->format(DATE_RFC850);
            $confirmRequest->date_closed = $currentTime->asDbDate();
            $confirmRequest->save();
            return true;
        }
        return false;
    }

}

 

The last file is our public custom entry point, which we install with the help of the EntryPointRegistry extension. This will allow us to create a publicly accessible URL on our Sugar instance that can be used to easily display HTML for a user readable response.

 

./custom/Extension/application/Ext/EntryPointRegistry/confirmEmail.php

 

<?php
// Copyright 2018 SugarCRM Inc.  Licensed by SugarCRM under the Apache 2.0 license.
$entry_point_registry['confirmEmail'] = array(
'file' => 'custom/src/ConfirmedOptIn/ConfirmEmailEntrypoint.php',
'auth' => false
);

 

Since this is a public entry point, we should take extra care to validate user input. We will use the InputValidation framework to assert that the URL parameters are both the GUIDs that we expect for record IDs.

 

./custom/src/ConfirmedOptIn/ConfirmEmailEntrypoint.php

 

<?php
// Copyright 2018 SugarCRM Inc.  Licensed by SugarCRM under the Apache 2.0 license.
if(!defined('sugarEntry') || !sugarEntry) die('Not A Valid Entry Point');

use Sugarcrm\Sugarcrm\Security\InputValidation\InputValidation;
use Sugarcrm\Sugarcrm\custom\ConfirmedOptIn\ConfirmedOptInManager;

$manager = new ConfirmedOptInManager();
$validator = InputValidation::getService();

$recordId = $validator->getValidInputGet('id', 'Assert\Guid');
$emailId = $validator->getValidInputGet('email', 'Assert\Guid');

if ($manager->confirm($recordId, $emailId)) {
    echo "<h1>Confirmed!</h1>";
} else {
    echo "<h1>Unable to confirm.</h1>";
}

 

This page will just show “Confirmed!” if the email address was properly confirmed and “Unable to confirm.” if the confirmation failed which could be due to badly formed request or because the email address or data privacy record IDs didn’t already exist in the system. You could update this entrypoint to return whatever HTML you want or perhaps even to redirect the person’s browser to some other page.

 

For example,

 

header("Location: http://www.example.com/success");

 

If you happen to see a blank page, then input validation has probably failed. You can check for a fatal message in sugarcrm.log for more details on the failure.

 

Step 4) Try it out!

 

Create a new Lead record that includes a new email address.

 

New Lead record

 

Switch over to Data Privacy module and you will find a new Data Privacy record like the one below.

 

 

For convenience, we can just click the confirmation URL itself to trigger the confirmation check of using an email to do it. In a real implementation, it may make sense to hide this URL from Sugar end users so they do not accidentally click on it since we expect someone who does not use Sugar to click it themselves.

 

 

Nice, it was confirmed! If the request isn’t quite right or the email address is not in the system then you will get the “Unable to confirm” message instead.

 

If I return to my Data Privacy record, then I can see it has been completed and that the email address on my original Lead record has been opted in.

 

 

There is a lot that could or should be done to enhance this example for production use. But you can see that the pattern is straightforward to implement using existing Sugar platform extensions and APIs.

While we introduced PHP 7.1 support in the Sugar 7.9.0 release back in May 2017, we have up until now continued to support PHP 5.6. But PHP 5’s days are numbered. We will be dropping support for PHP 5.6 in the Sugar 8 and Spring ‘18 releases. Sugar 8.0.0 and Spring ‘18 will only support PHP 7.1. Long live PHP 7!

 

In the Sugar cloud, we have been migrating from PHP 5.6 to PHP 7.1. With this transition, we have discovered some additional PHP 7 pitfalls that we want to make sure everyone avoids in their Sugar customizations. These problems affect any version of Sugar that uses PHP 7. They are both centered around a new ArgumentCountError that was introduced in PHP 7.1.

 

If you haven’t already, you should check out the old blog post we did on preparing customizations for PHP 7. There are lots of benefits for using PHP 7 with Sugar. Not only are there performance improvements but Sugar code as well as customizations are now able to take advantage of more advanced PHP language features.

 

Let’s take a look at the two pitfalls we want to help you avoid.

 

Do not use deprecated PHP 4 style constructors in Logic Hooks classes

You should not be using PHP 4 style constructors in PHP 7 at all, but there is an additional complication that is introduced when you are using Logic Hooks with PHP 7.1.

 

Take the following example:

 

class Hook {
    public function hook($bean, $event, $arguments)
    {

    }
}

 

The issue here is rooted in the fact that functions and classes in PHP are case insensitive.  The hook() function contained within the Hook class is interpreted as a PHP 4 style constructor even though the names differ in case.

 

For logic hook code where the class name and function name differ only in case, it would cause Sugar to throw fatal PHP errors in PHP 7.1 and warnings in earlier PHP versions. Since the class name and function name does not match in the example, Sugar will not recognize hook() as a constructor. So Sugar will first instantiate the Hook object and then subsequently call the hook() function later. But PHP recognizes hook() as a constructor. So PHP would instantiate a Hook object without passing the needed arguments to hook(). Calling a function with an incorrect number of arguments in PHP 7.x throws a fatal ArgumentCountError.

 

When using PHP 5 with Sugar, this would generate a PHP warning and ultimately the hook() function would be called twice; once with no arguments for the constructor and then a second time with arguments to process the hook.

 

So there’s all kinds of things going wrong here in this very brief example.

 

For compatibility reasons, we have fixed this case sensitive behavior where we will only call this function once in Sugar 8.0.0 but even then you are still using a deprecated PHP 4 constructor in PHP 7. This is either unintentional or simply a bad idea.

 

You must use function names that are different than your class names in your Logic Hook classes.

 

We can easily fix the example above by changing the function name.

 

class Hook {
    public function afterSave($bean, $event, $arguments)
    {

    }
}

 

 

Do not define too many arguments for Logic Hooks functions

Now we should take a look at another issue that will throw an ArgumentCountError.

 

class Hook {
    public function afterSave($bean, $event, $arguments, $customArgs)
    {

    }
}

 

The developer who wrote the afterSave() function above may have been trying to save time by adding the $customArgs argument to a standard after_save module logic hook function to handle some alternative usages. The issue is that Sugar is going to call this logic hook function with three arguments instead of four, which will throw an ArgumentCountError in PHP 7.1. In PHP 5, a warning is simply logged and the function is still executed.

 

It should go without saying that functions should be called with the correct number of arguments.

 

¯\_(ツ)_/¯

 

We can fix the example above to ensure that functions can always be called with the correct number of arguments.

 

class Hook {

    public function afterSave($bean, $event, $arguments)
    {
        $this->otherSave($bean, $event, $arguments, array());
    }

    public function otherSave($bean, $event, $arguments, $customArgs)
    {

    }

}

 

You must ensure that your Logic Hook functions use the correct number of arguments.

 

For most logic hook events that is three arguments but sometimes it is two! Check the event definitions in the Logic Hooks section of the Developer Guide if you aren’t sure how many arguments are required!

 

Recap

Let’s recap what you should do to prepare your Logic Hooks for PHP 7.1:

  • Ensure that none of your Logic Hook classes contain functions have the same name as the class. Remember that PHP is case insensitive when it comes to class and function names.
  • Ensure that your Logic Hook functions use the correct number of arguments.

Hi there, Sugar Developers!

 

We want to make sure your customizations and integrations are ready for the Sugar 8 and Spring ‘18 releases, so we’re hosting two webinars just for you!

 

Webinar Topics

We’ll discuss the big changes that are likely to impact you including the following:

  • Review Data privacy features used for GDPR compliance (if you’re not thinking about this, you probably should be!)
  • New Sidecar components and features that support Data Privacy
  • New Admin panel to register custom API platforms
  • Changes to Audit APIs, data model, and behavior
  • Changes to supported platforms and libraries
  • Update on the Professor M School

 

Webinar Information:

We are holding 2 sessions to accommodate various geographical locations. Please choose ONE of the following times below.

Monday, April 23nd 6:00 PM - 7:00 PM PT

OR

Tuesday, April 24rd 8:00 AM - 9:00 AM PT

(Choose one)

 

Register Here


We will be posting the webinar recordings here in the Developer space for those who are unable to attend the live sessions.

Please don’t hate me. It’s not really my fault. I am sorry to say that every single Sugar developer needs to care about the General Data Protection Regulation (GDPR) and data privacy in general. It is a sign of the times and part of the interrelationship of technology and modern society that impacts the software we implement today and in the future.

 

I can explain but first let me provide some context.

 

What is the General Data Protection Regulation (GDPR)?

 

First off, I am not a lawyer or a GDPR specialist. If you really want to understand what GDPR is and what steps you need to take to be compliant, then I suggest reading the Regulation for yourself or hiring counsel. GDPR is a complex set of rules, and there’s still a lot of debate about what it all really means. Other interpretations, including yours, may be different from what I describe below.

 

Many of our colleagues in the European Union are well acquainted with the GDPR which requires full compliance by May 25th, 2018. But for those of you who do not live in the European Union (EU), I highly recommend reading Data Privacy and GDPR in Sugar by Deepak Deolalikar. It covers some GDPR basics as well as provides an overview of some of the features planned in Sugar to help our customers comply with GDPR.

 

In short, the spirit of GDPR is about allowing individuals (data subjects) to have control over their own personal information. GDPR ensures that personal information or Personally Identifiable Information (PII) of EU citizens are processed responsibly. EU citizens have a right to privacy that in many cases means you must comply with their requests to access, restrict the use of, or delete the personal information that is stored about them.

 

The scope is broad and the penalties are severe. Any organization that stores personal information about EU citizens, even if that organization is based outside the EU such as in the United States, is subject to this regulation.

Penalties for failure to comply with these regulations could result in fines that start at €20 Million or 4% of total revenue if that value is greater.

 

Are you paying attention now?

 

Data Privacy is yet another software globalization requirement

 

When we build software, we make sure that it meets all sorts of requirements. We design for performance, scalability, and security. We work hard to ensure a positive user experience and high quality software. At SugarCRM, we also focus on making sure that all our products are ready for the global marketplace. That means that over the years we have invested in translations into dozens of languages and support for multiple currencies, numerous date and time formats, right-to-left languages, Unicode character encodings (though we’re still working on emojis), and 508 compliance.

 

The fact is that GDPR is just one example of a data privacy regulation at a time when there are dozens more government bodies all over the world considering and implementing new data privacy regulations. UTF-8 character encoding and a translatable UI are no longer enough to make sure your software is ready for the global marketplace! The responsible collection and processing of personal information is now an additional obligation for all software developers.

 

With the upcoming Sugar Spring ‘18 (cloud) and 8.0.0 (on premise) release, we are now investing in data privacy features. These features will help provide our customers with the tools they need to comply with GDPR and many other data privacy rules or regulations.

 

These data privacy features and other data privacy concerns will impact Sugar customizations and integrations that collect or process personally identifiable information (PII).

 

Future installments on Data Privacy and GDPR

 

We will be exploring several data privacy topics in the coming weeks. Here is a quick summary.

 

Collecting explicit consent before storing data in Sugar

 

GDPR has strict requirements for consent. Unless you have an existing lawful basis for using personal information, you need to collect positive and unambiguous consent from the data subject even before data is stored in Sugar. For example, a web to lead form with a pre-selected opt-in checkbox is not going to cut it in the EU. We will be exploring ways to make sure you are collecting explicit consent using techniques like double opt-in (DOI) or confirmed opt-in (COI).

 

Managing PII data and the right to erasure in Sugar

 

There are new Sugar APIs being added for working with PII as well as features that allow Sugar customers to comply with data subject requests like the right to access their own data and the right to erasure. In particular, external systems that integrate with Sugar may need to identify and implement their own erasure measures in compliance with a data subject request.

 

Improved change log and attribution of changes using Data Sources

 

We are implementing some enhancements in Sugar’s change log functionality to allow for finer grain tracking and proper attribution for changes to PII. We will be exploring these platform level enhancements including the ability to define new data sources (for example, an integration) that are responsible for changes to data.

 

What do you think?

 

What are some of your biggest data privacy or GDPR concerns? Let us know and we’ll try to address them in future posts.

 

Also, please follow this space and this blog to be sure you are notified when the next installment is posted!

What is the Sugar Metadata API?

As you probably know, Sugar metadata encompasses the settings, data model (Vardefs), and visual layouts (Viewdefs) used by the Sugar application. The majority of changes made to Sugar using Sugar Studio, Module Builder, or even custom code are implemented by modifying metadata.

 

How can I know that the custom field or custom modules that you need for your integration to work exists in Sugar?

How can I know what are the allowed values are valid for a Sugar dropdown field?

How can I know if a field is required?

How can I know the display label for a given field?

How can I know what fields are available in Sugar that I can map back to an external data source?

 

The answers to all these questions and many more can be found in Sugar metadata.

 

On the server side, metadata is implemented as an extensible set of PHP associative arrays. But how does this information get sent out to Sugar clients like your web browser, mobile app, or even the Outlook plug-in? This is accomplished via the Sugar metadata endpoint of the Sugar REST API.

 

GET /rest/v11/metadata HTTP/1.1
Host: localhost:8080
OAuth-Token: ...
Content-Type: application/json

 

The full metadata response comes back as a very large JSON object. It's usually a few megabytes in size (about 3MB in a stock version of Sugar Winter '18) but can be much larger depending on customizations. Make sure compression is enabled on your web server if you are going to be using it over a slow connection!

 

 

The value to using the Metadata API is that you can discover changes in Sugar configuration and Sugar customizations. New custom fields, relationships, modules, changes to Sugar views and layouts, and much more are all represented in Sugar metadata responses.

 

From a REST API perspective, the only way to accurately know the current data model for any given Sugar instance is to use the Metadata API.

 

Since the Metadata API is so important, here are three important tips to keep in mind to make development easier.

 

Tip 1: Retrieve only what you need

Metadata API calls can generate a huge response depending on your Sugar instance configuration and the customizations that may exist. You should specify filters on your metadata request in order to narrow the response to the data that you actually need. Narrowing what is returned will improve performance on both server side in terms of generating the response but also on the client side where you need to parse the JSON response.

 

For example, if you are integrating with only the Accounts module then you may only need to concern yourself with that part of the metadata. Specify a type_filter and module_filter as URL parameters to only return module metadata for the Accounts module. Other supported type_filter options are server_info, config, relationships, labels, languages, currencies, views, layouts, full_module_list, and etc.

 

GET /rest/v11/metadata?type_filter=modules&module_filter=Accounts HTTP/1.1
Host: localhost:8080
OAuth-Token: ...
Content-Type: application/json

 

Tip 2: Cache metadata, update only when it changes

You can safely cache the metadata that you need if you keep track of the _hash value returned in the metadata response.

 

You should present that hash value in the X-Metadata-Hash header on subsequent Sugar REST API requests to ANY endpoint. If the metadata hash is out of date then Sugar will return a 412 PRECONDITION FAILED HTTP response. You should then retrieve updated metadata from the Metadata API and store the updated metadata hash value.

 

GET /sugar/rest/v11/Accounts HTTP/1.1
Host: localhost:8080
OAuth-Token: ...
Content-Type: application/json
X-Metadata-Hash: 705fb99230e2ec60b59f481c6831a7cf

 

Also worth mentioning, many of our API resources (such as records and our metadata) implement HTTP caching. If an API response contains an E-Tag HTTP header then you can safely cache the response body as well as the E-Tag value. If you present this value in the If-None-Match HTTP header properly on a subsequent HTTP GET request for that resource, you will get an empty 304 NOT MODIFIED response back if it is safe to continue using the cached version of the previous response. This is another approach to reduce needless processing.

 

GET /rest/v11/metadata?type_filter=modules&module_filter=Accounts HTTP/1.1
Host: localhost:8080
OAuth-Token: ...
Content-Type: application/json
If-None-Match: 705fb99230e2ec60b59f481c6831a7cf

 

This allows you to update your local copy of the metadata as soon as it changes and precisely no more often than that.

 

Tip 3: Different users and platforms will retrieve different responses

If a user is assigned a role that doesn't have access to a particular module then the metadata for that module will not be returned when that user accesses the metadata API. Makes sense, right? 

 

Sugar also support things like role based views which will provide alternative UI layout metadata based upon the current user's role.

 

Different platforms will have different metadata responses as well. The mobile platform retrieves module metadata that reflects the needs of our Sugar Mobile app. Mobile UI components look a lot different from what you see when you access Sugar from your browser. The metadata that is retrieved is significantly different to accommodate these changes.

 

For example, the mobile client uses separate edit and detail views for representing records instead of a single record view like you find in Sugar web client. This is represented in the mobile metadata response.

 

What is the Input Validation Framework?

The Input Validation Framework is a centralized security framework used to validate that external input is provided in the expected format. Validating user input is critical because it is the basis for generated HTML and REST API output, used in the construction of SQL queries, used to access files, etc. Malicious input can be used to compromise the security of a Sugar instance and the underlying infrastructure. The validation framework provides better control than the pre-existing input sanitization techniques applied to user input within Sugar.

 

Fixing Input Validation problems

Sugar’s input validation was enabled but only logging failures as warnings into the sugarcrm.log file since the 7.7.1 release. With the Winter ‘18 release, the validation.soft_fail configuration setting will now default to false. This means that input validation checks will be strictly enforced by default instead of simply logging warning messages.

 

For more information about using validation constraints in your own custom code, check out the Sugar Developer Guide, the previous UnCon Security presentation, or the Winter '18 webinar.

 

Identifying input validation violations

Beginning in the Winter ‘18 release, input validation violations are reported as PHP Fatal errors in the PHP error log and Critical error messages in sugarcrm.log file. Invalid requests will generally be terminated with an HTTP 500 error as well.

 

Let’s walk through an example of an input validation violation.

 

In this example, we have a customization that is accessing the Sugar instance at the following URL: sugarcrm/ult/#bwc/index.php?module=xxxxx.

 

If xxxxx is not a registered module, there will be a FATAL error in the PHP error log.

For example,

[15-Jan-2018 22:04:09] WARNING: [pool www] child 1940 said into stderr: "NOTICE: PHP message: PHP Fatal error:  Uncaught exception 'Sugarcrm\Sugarcrm\Security\InputValidation\Exception\ViolationException' with message 'Violation for REQUEST -> module' in /srv/www/<instance>/prebuild/test/ult/sugarcrm/src/Security/InputValidation/Request.php:280"
[15-Jan-2018 22:04:09] WARNING: [pool www] child 1940 said into stderr: "Stack trace:"
[15-Jan-2018 22:04:09] WARNING: [pool www] child 1940 said into stderr: "#0 /srv/www/<instance>/prebuild/test/ult/sugarcrm/src/Security/InputValidation/Request.php(209): Sugarcrm\Sugarcrm\Security\InputValidation\Request->handleViolations('REQUEST', 'module')"...

 

The sugarcrm.log will also show an error.

For example,

Tue Jan 16 06:05:18 2018 [1941][-none-][CRITICAL] InputValidation: [REQUEST] module -> Invalid module xxxxx

 

From the end user perspective, these fatal errors will cause pages to appear blank or stalled during loading. If you inspect the browser console when this happens, you will likely see HTTP errors logged.

 

Fixing input validation violations

The steps to take to fix the violation are going to depend on the exact constraint that you are violating. Luckily the exceptions you find in the PHP error log and the messages in the sugarcrm.log will indicate the constraint that you are violating.

 

In the above example, attempting to access a BWC view for an unregistered custom module “xxxxx” is causing the failure. If you make sure that the module you are using is properly registered in Sugar’s module list, or modify your request to use an existing Sugar module, then this will fix this issue since the constraint is checking if the module name is a known modules installed in this Sugar instance.

 

There are basic Symfony framework PHP constraints that we use. These do not need an in depth explanation. The basic Symfony constraints can be used to validate URLs, date time formats, etc. Check out the Symfony documentation for these basic constraints.

 

In Sugar's implementation, we add our own set of validation constraints. These constraints extend from Symfony's Constraint and ConstraintValidator classes and are located in the ./src/Security/Validator/Constraints directory of your Sugar installation. Each Constraint is enforced using a related Validator class. You can inspect the Validator classes to see the precise logic that needs to be followed. For example, the Bean/ModuleName constraint is implemented by the Bean/ModuleNameValidator class. This constraint is used to ensure that the input is a string that matches the name of a valid module with an associated SugarBean class.

 

One of the basic rules that most of these constraints implement is that an input type should be a scalar. This means we expect most user input to be integer, decimal, string, or boolean values.

 

Sugar Constraints

The list provided below may not be exhaustive.

  • Bean/ModuleName
    • Must be a string. The Module’s SugarBean class must be retrievable using BeanFactory. For example, this is used in legacy Workflow.
  • ComponentName
    • Must start with a letter and may only consist of letters, numbers, hyphens and underscores. For example, this is used throughout Studio and Module Builder for module names, package names, and Sugar Logic functions.
  • Delimited
    • By default, must be comma delimited scalar values. For example, this is used in conjunction with other constraints.
  • DropdownList
    • Dropdown lists must only consist of letters, numbers and underscores. For example, this is applied to dropdown lists during module install.
  • File
    • File must exist in an allowed file path and must not include null characters or any directory traversals. For example, this is used by FileLoader utility class.
  • Guid
    • Must be a string with alphanumeric or dash characters. For example, this is used with SugarMVC Views.
  • InputParameters
    • General input validation for GET, POST, REQUEST superglobals. Must be scalar values and strings should not contain null characters.
  • JSON
    • Must be a string that is JSON decodable. Returns decoded PHP associative array. For example, used in Configure Navigation Bar Quick Create.
  • Language
    • Must be a locale code that matches an installed language. For example, this is applied where languages can be selected in UI.
  • LegacyCleanString
    • Reimplements input sanitization rules previously used with clean_string() function.
  • Mvc/ModuleName
    • Must be a string. A more permissive test than Bean/ModuleName that allows any module name that can be found in global module lists. For example, this is used throughout SugarMVC Views.
  • PhpSerialized
    • Must be a PHP serialized string that does not contain objects. Returns unserialized value.
  • Platform
    • Must be a string that is under 128 characters and contains alphanumeric and dash characters. Must be a known Platform value such as “base”, “mobile”, “portal”, etc., or one registered using Platform extension. For example, this is used to validate REST API login requests.
  • Sql/OrderBy
    • Must be a string that represents a valid ORDER BY SQL clause. For example, this is used in SugarMVC list views.
  • Sql/OrderDirection
    • Must be a string that is uses SQL keywords “ASC” or “DESC”. For example, this is used in SugarMVC list views.
  • SugarLogic/FunctionName
    • Must be a string that only uses word characters and dashes. For example, this is used in Sugar Studio.

 

If you encounter a failure, then you should be able to identify the constraint that you are violating from your error logs. By looking at log messages that appear immediately before the failure, and possibly some exploration in the Sugar user interface, you will be able to identify the failing request. From there, you can use the information above to alter the request to comply with the necessary constraints.

 

This is not a substitute for Sugar field level validation!

An important thing to note about the Input Validation Framework is that it is not the only input validation mechanism available. For example, there is still separate field level validation that occurs when users create or edit records. Sidecar customizations or Studio configuration can still be used to perform validation of user provided input for each field of a record in a user friendly way. For example, Sugar will let you save a record without filling out required fields and will display a user friendly warning that tells user what to correct. The Input Validation Framework is centered around security and does not provide the same usability than you would get validating user input using Sidecar or Studio settings.

The Sugar Winter '18 release is fast approaching! Hopefully you are as excited about the new Sugar UX as I am!

 

As usual, Lauren Schaefer and I will be hosting a developer webinar to help get you and your customers prepared for the new Winter '18 (7.11) release. Please read below to learn more about the agenda and do not forget to register! We run these webinars twice, so no matter where you are in the world, we hope that one of these times will be convenient for you.

 

Winter '18 Technical Webinar Agenda

 

Technical Training AgendaWebinar Times
  1. Impacts of new interface
  2. Sugar Exemplar app (more below)
  3. Restricted REST API platforms (more below)
  4. Removal of some LESS files
  5. Emails subpanel changes
  6. Input Validation Framework (now the default)

Monday, January 22nd 4-5:00 PM PT
Tuesday, January 23rd 8-9:00 AM PT

Register Here

 

We will be posting the webinar recordings and deck to the Partner Portal for those who are unable to attend the live sessions.

 

Sneak peak of Sugar Exemplar app


The new Sugar Exemplar app is designed to demonstrate best practices for developing Sugar customizations (think DevOps and QA!) and a solid example of what upgrade-safe customizations should look like. We’ll tell you more about the app and how you can get involved during the webinar!

 

Reminder about restricted REST API Platforms


ISVs and developers who use a platform string to connect to a Sugar instance will need to update their existing API integrations. For more information - please refer to this blog post on how to to proceed.

 

Get on our developer mailing list!

 

Remember to opt-in for Sugar Developer communications to make sure you don't miss this and other important updates.

Sugar v11 REST API

In order to support some cool new dashboard features, we made some enhancements to our Dashboard APIs in the Fall '17 release. So we added a new REST API version, v11, in that release in order to ensure clients and integrations that were using the existing v10 Dashboard APIs would continue to work properly. For more details on the v11 API, we recommend you check out a recording of our recent Fall '17 release developer webinar and the related migration guide.

 

Today we won't focus on version 11 specifically but instead on how the Sugar platform will be handling REST API versioning in the future. So read on!

Passing API version using HTTP Header

For REST v10 and our older web services frameworks, the API version was specified exclusively via the URL.

 

Example 1)

 

GET <sugar>/rest/v10/me

 

In the Fall '17 (7.10) release, we introduced an additional way to specify the API version: using an HTTP header. This change allows resource URLs to be treated as unique identifiers to resources regardless of the API version in use.

 

The REST API version can now be specified via HTTP Accept header. The REST API version should be specified in either the URL or in the HTTP Accept header but not in both locations at once. This new method of specifying the API version in the HTTP Accept header better adheres to RESTful architectural principles. By essentially encapsulating the API version as part of media type, a single resource at a given URL could offer multiple representations using different data formats and API versions.

 

Example 2)

 

GET <sugar>/rest/me
...
Accept: application/vnd.sugarcrm.core+json; version=42

 

We recommend that client start specifying the API version in the Accept header instead of as a component of the URL. Sugar clients will adopt use of the Accept header in the future.

Future use of semantic versioning in REST API

As you probably have noticed, the REST API version is a property of the Sugar REST API as a whole and not individual endpoints.

 

We plan to implement a semantic versioning policy with MAJOR.MINOR format for the REST API next year.

 

The version will be specified in the request Accept header as MAJOR.MINOR (ex. 11.1) whereas the version in the endpoint url will be specified as MAJOR_MINOR (ex. 11_1).

 

The Sugar REST API MAJOR version will be incremented for each Sugar release where compatibility is being broken on any endpoint. The Sugar REST API MINOR version will be incremented for each Sugar release where an API endpoint is changed or additional API endpoints are being added without breaking compatibility.

 

Not every new Sugar release will result in new REST API versions. Our intent is to only add new features or make breaking changes to our REST API only when it is not possible or prudent (ex. for performance reasons) to do so with the current API.

 

When there is a breaking change, older API versions should continue to work as-is. For example, if a parameter was removed in newer version, the old version should still support that parameter.

What are breaking and non-breaking changes?

We want to be clear about what we consider to be an API breaking change and what is not.

Breaking Changes

  • Removing or renaming an endpoint, a request parameter, or a fixed response field
    • For example, removing the GET /rest/v10/<module> endpoint or renaming the _acl field to _access_control.
  • Removing support for a HTTP request method on an endpoint
    • For example, changing from PUT to PATCH method for updating records.
  • Change in type for a request parameter
    • For example, a change from an Integer to Decimal value.
  • Change in response field format
    • For example, a change in encoding for a response field from plaintext to base64.
  • Semantic changes to request parameters
    • For example, a change from treating the % symbol as a literal instead of as a wildcard or a change in default value.
  • Semantic changes to an API endpoint
    • For example, an endpoint that was documented to return all records by default and now returns no records by default.

Non-breaking Changes

  • Metadata related changes in behavior
    • For example, when Sugar metadata (Vardefs, Viewdefs, etc.) changes are introduced during a Sugar release or as a result of a customization then REST API responses can vary as a result.
  • Adding a new endpoint
    • For example, adding a GET rest/Accounts/:id/map endpoint.
  • Adding a parameter to a request
    • For example, adding a boolean parameter called shorten_text that decides if long text fields are shortened to 200 characters.
  • Adding a field to a response
    • For example, adding a _self field with link to resource URI in GET responses.
  • Adding support for another HTTP request method
    • For example, adding support for PATCH method to an endpoint.

 

If you ever encounter a situation where there is a breaking change on a particular API version between future Sugar releases, then please report those issues via the Sugar Support portal.

Deprecation process for old API versions

Old REST API versions will continue to be supported until at least one year after they have been deprecated. We want to exercise good judgement on how quickly we remove old API versions but please understand that SugarCRM cannot indefinitely support an unlimited number of REST API versions.

What should you do?

  1. When building new Sugar clients or integrations or updating existing ones, adopt the latest available REST API version possible in order to take advantage of all the latest features.
  2. Use the HTTP Accept header to specify REST API version when using Sugar Fall '17 release or later.
  3. Monitor the release notes and documentation on each new Sugar release for deprecation notices on old API versions. Keep our customers safe by planning ahead!