Skip navigation
All Places > Developer > Blog
1 2 3 Previous Next

Developer

165 posts

Hey there, developers!

 

We are hosting lunchtime table topics just for developers on both days of SugarCon.  After you pick up your lunch, you'll be able to browse tables that are labeled with various topics and select one that interests you.  We'll have a facilitator at each table to get the conversation going.  Everyone, regardless of experience level, is invited!  You're welcome to share your knowledge, ask questions, or just sit and listen.  This is your chance to do a little networking with other developers and learn something that may not be covered in the breakout sessions.

 

We need your help in deciding what the table topics will be!  I've added comments to this blog post with some ideas we have.  If we missed a topic you're interested in discussing, add a comment below with your suggestion.

 

Most importantly, we want you to vote!  Click the Like button on topics below to vote for your favorites.

 

We can't wait to see you at SugarCon on October 10 and 11!  If you haven't registered yet, you can register at sugarcon.com

 

P.S.  Don't forget to submit your proposal to give a 5 minute lightning talk at SugarCon by Tuesday, August 21, 2018!

Or maybe you will.  Click-bait headlines are the worst.  Don't worry, I won't make you click through an ad-covered slideshow to find all of the library changes.

 

Anyway, we have a few library updates coming in our upcoming Fall '18 release, and we wanted to give you as much time as possible to begin preparing for the changes.  Without further ado, below is information on upgrades to Handlebars, Underscore, jQuery, jQuery Migrate, and jQuery UI.

 

Handlebars

The Handlebars library is a JavaScript library that lets Sugar create semantic templates.  Handlebars help render content for layouts, views, and fields for Sidecar.  You can learn more about Sugar's use of Handlebars in the Sugar Developer Guide.

 

Handlebars v4 included significant changes to depthed paths that would likely break many customizations and integrations (see the Handlebars 4.0.0 Release Notes for more details).  However, v4 also included improvements we wanted to get to you.  Below is information on our solution.

What is changing?

In order to get you the improvements without breaking your customizations and integrations, we have created a forked version of Handlebars v1.3.0 with a unique version number ("1.3.0-sugarcrm-temporary") that we intend to include in the Fall '18 release. 

What documentation should I reference?

What's coming in the future?

The forked version of Handlebars is intended to be a temporary solution, and customizations should NOT rely on this fork.  Given the breaking changes to depthed paths in v4, we are exploring alternatives to reduce disruption to existing customizations and allow developers as much time as possible to adapt. Watch this blog for updates as we plan our path forward.

 

Underscore

Underscore is a Javscript library that Sugar uses to improve code readability and performance.  You can learn more about how you can use Underscore in your apps and customizations in Matt's blog post Using Underscore.js to improve Sugar 7 performance.

 

Underscore 1.9.0 includes big performance improvements that we want you to benefit from, so we will be upgrading it.

What was upgraded?

  • Underscore will be upgraded from 1.8.3 to 1.9.1.
  • In Ventana, Underscore was upgraded from 1.8.3 to 1.9.1.

What documentation should I reference?

What else should you know?

No functionality will be deprecated as part of this upgrade, so we expect the impact to be quite low.

 

jQuery, jQuery Migrate, and jQuery UI

We are upgrading jQuery, jQuery Migrate, and jQuery UI in the Fall '18 release.  jQuery is a Javascript library we use for tasks like traversing and manipulating the DOM and handling events.  jQuery Migrate is a Javascript library we use to help us migrate to newer versions of jQuery without our code breaking due to APIs being removed in the newer versions.  jQuery UI is a Javascript library we leverage for various interface interactions, effects, widgets, and themes.

 

Below are (hopefully) all of the answers to your questions related to the upgrades of these libraries.

What will be upgraded?

  • jQuery will be upgraded from 1.11.3 to 1.12.4 for both Sidecar and BWC. 
  • jQuery Migrate will be upgraded from 1.2.1 to 1.4.1 for both Sidecar and BWC.
  • jQuery UI will be upgraded from 1.11.4 to 1.12.1 for both Sidecar and BWC.
  • In Ventana, jQuery was upgraded from 1.11.3 to 1.12.4 and jQuery Migrate was upgraded from 1.2.1 to 1.4.1.

What will be removed or renamed?

  • The new versions of these libraries remove and deprecate various functions.  See the Changelogs and Release Notes in the Documentation section below for more details.  Two changes we think may affect many of you:

    • jQuery UI's .zIndex() function was removed.  You can consider using the .css() function instead.  For example, if you used .zIndex() to get the z-index of an element, you could use .css('zIndex')instead.  In cases where you set the z-index of an element by doing something like .zIndex(1), you could use .css('zIndex', 1)  instead.  Be sure to test your changes to ensure everything is behaving as you expect.  For other recommendations on how you could update existing code that uses the .zIndex() function, see the jQuery UI 1.12 Upgrade Guide
    • jQuery UI's CSS class name ui-icon-carat was renamed to ui-icon-caret.  We recommend searching and replacing all instances of ui-icon-carat with ui-icon-caret.
  • jQuery version 1.7.1 will be removed; portal/lib/jquery-ui/js/jquery-1.7.1.min.js will be removed.
  • An unused copy of jQuery 1.6.4 that was available at include/javascript/jquery.js will be removed.
  • The jQuery UI CSS component files will move to a components directory and be renamed to no longer begin with jquery.ui.  For example, the jQuery UI dialog was previously include/javascript/jquery/themes/base/jquery.ui.dialog.css and will be moved and renamed to include/javascript/jquery/themes/base/components/dialog.css.

What will not change?

For backwards compatibility, we will retain the file include/javascript/jquery/themes/base/jquery.ui.all.css and it will continue to work as before. Image files in include/javascript/jquery/themes/base/images will also be retained.

What documentation should I reference?

What's coming in the future?

Upgrading jQuery and jQuery Migrate to version 3.x is on the roadmap for an upcoming release, so we recommend beginning to look at how you will update your customizations and integrations to use a modern version of these libraries.

What else should you know?

A minified version of jquery.ui.all.css will be available at include/javascript/jquery/themes/base/jquery.ui.all.css.

 

If you are using jQuery plugins (see The jQuery Plugin Registry or The npm Package Search), you may need to upgrade those to newer versions that are compatible with jQuery 1.12.4 and jQuery UI 1.12.1.

We're doing something new at SugarCon this year, and we want you to be a part of it!  During happy hour on the first night of the conference, we're hosting a series of lightning talks for our developers. 

 

These lightning talks are going to be five minutes long, consisting of 20 slides that automatically advance every 15 seconds.  You can learn more about this exciting talk format in the video below!

 

Whether you're brand new to public speaking or a seasoned speaker, this is a great way to share your story, add to your speaker's resume, and build your personal brand.  Attendees will be sipping cocktails and ready to cheer you on! 

 

Our call for proposals is open now until Tuesday, August 21, 2018.  Submit your ideas here

 

We look forward to hearing your story!

This blog by SugarCRM Technical Account Manager and Solution Architect Angel Magana is reposted with his permission. See Angel's Blog: SugarCRM: Related Data Interactions for the original post from Angel's blog. In this post, Angel explains some best practices for working with related records in PHP that can help improve performance of your customization.

Fetching related records

 

Often times a customization we are working on requires us to interact with related data, usually records that have a parent-child relationship with one another. For example, a customization might require us to determine if a given Account record has 1 or more related Contacts. Or for that matter, we may need the collection of ID values that represent all the Calls linked to a Contact.

 

In either example, the path most often followed could be described with the following pseudo-code:

 

1. Instantiate the parent SugarBean object (e.g. Account/Contact record object)
2. Load the corresponding relationship to access the related data objects
3. Retrieve the related SugarBean objects representing the related objects

 

See below for a PHP snippet illustrating the above:

 

<?php 

//Some other code

//Instantiate parent record SugarBean object
$ParentBean = BeanFactory::retrieveBean('Accounts', 'some_account_id_value');

//Load the pertinent relationship using Link field name
$ParentBean->load_relationship('contacts');

//Retrieve related Contact(s) SugarBean object(s)
$RelatedContacts = $ParentBean->contacts->getBeans();

//The rest of our code


Line 12 in the above example would effectively create an array of SugarBean objects representing the linked Contacts. From that point forward, determining whether or not the Account has any linked Contacts becomes a simple matter of checking the size of the array. 

 

For the linked Calls example, the code would be very similar, except we would load the 'calls' relationship and the ID values we want would be in the SugarBean object array that results from executing the getBeans() method.

 

Do you need full Beans or just the IDs?

 

Now, all of the above would function just fine, but let us consider a few things about this approach.

 

First, upon executing the getBeans() method, we are asking Sugar to retrieve the complete SugarBean object for all the related records of a given type. However, for both examples, simply retrieving the ID value would have sufficed.

 

That is, for our purposes, a list of ID values in the $RelatedContacts array is just as good as an array of all the SugarBean objects. That array would still allow us to properly answer the question of whether there are or are not any linked contacts, solely based on the number of ID value entries the array contains.

 

In the case of the second example, we specifically only need the ID values of the related Calls, but getBeans() gives us everything pertaining to each of those calls. This tells us there is a lot of overhead we can trim.

 

From a performance standpoint, anytime we can optimize our code to retrieve only what we need, we help minimize wait times a user experiences. 

 

How do we then change our code to only give us the ID values?

 

It is actually a quite simple.

 

<?php 

//Some other code

//Instantiate parent record SugarBean object
$ParentBean = BeanFactory::retrieveBean('Accounts', 'some_account_id_value');

//Load the pertinent relationship using Link field name
$ParentBean->load_relationship('contacts');

//Retrieve related Contact(s) IDs ONLY.
$RelatedContacts = $ParentBean->contacts->get();

//The rest of our code

 

Examining the Link2 class in Sugar reveals a get() method which specifically only returns ID values of related data. Using get() in place of getBeans() would allow us to achieve the goal described in our examples and also reduce the performance overhead.

 

Side note: A similar method exists in the TeamSet class, named getTeamIds(), to retrieve the ID values of Teams that compose a TeamSet, without needing to retrieve all of the Team SugarBean objects. Also be aware that get() will return all records, irrespective of team assignments on the related records. 

Hey there, developers!  We've officially released Summer '18 !  This is our fourth quarterly release for Sugar Cloud (formerly known as Sugar On-Demand).  Can you believe we now have a full year of quarterly releases?  

 

This release has a ton of great features including an updated login page, built-in support for double opt-in (think GDPR and data privacy!), emoji support , over 100 new built-in reports, and big improvements to advanced workflow.

 

Our Co-Founder & CMO, Clint Oram, discusses the highlights of this release from an end-user's perspective in the video below: 

 

 

Matt Marum and I recently hosted a webinar where we gave an overview of the big things developers need to know about this release: 

 

The slides from the webinar are available here.

 

If you're looking for the high-level overview, I've got you covered!

  • We have updated the Login page to display new content, so you'll want to make sure any customizations you made to the Login page still work.
  • Sugar now supports emoji!  You can get the details in Matt's blog post  
  • The GDPR and data privacy are hot topics, and that's why we have added to our existing data privacy features to support double opt-in.
  • We created a new ReportSchedules module that utilizes Sidecar.  If you have customized the old BWC report scheduling view, you'll need to reimplement those customizations for the new ReportSchedules module.
  • You can now export and import all business rules and email templates used in a process definition.  You no longer have to worry about manually reconfiguring pieces of your process definition when moving between environments.
  • We will be posting this release in the Developer Builds community so you can do your development locally.

 

This blog post promised to have just about everything you need to know about the Summer '18 release.  Below are some resources that have the rest of the details.

 

We hope you’re as excited about this release as we are!

Hello Sugar Developers!

 

We want to make sure your customizations and integrations are ready for the upcoming Sugar Summer ‘18 release, so we’re hosting two webinars just for you! The Summer '18 release is just a couple short weeks away.

 

What we will be covering:

We’ll discuss the big changes that are likely to impact you including the following:

  • Changes to Login screen and authentication classes
  • Emoji support
  • Built-in support for Double Opt-In (DOI)
  • Improvements to Reports and Advanced Workflow
  • SugarCon Update

 

Webinar Information:

We are holding 2 sessions to accommodate various geographical locations. Please choose ONE of the following times below.


Monday, July 2nd, 5:00 - 6:00 PM PT 
OR 
Tuesday, July 3rd 8:00 - 9:00 AM PT 
(Choose one)

 

Register Here

 

We will be posting the webinar recordings to Developer community for those who are unable to attend the live sessions.

 

P.S. Did you know you can apply for free passes to SugarCon by speaking at the conference or becoming a Sugar Scholar? Get the details on Two fabulous ways to earn a FREE pass to SugarCon 2018.

We have hit a tipping point in world wide Emoji usage. We really must work like we live, because the perfect two character combination of pizza and beer and others like it are starting to appear in business emails and other business communications. In particular, marketers have started to use emojis in subject lines of emails as a way to catch people's attention.

 

As Rich Green put it:

It's nearing the end of the second decade of the third millennium.

If people want to use poop emoji's in business, so be it.

 

Many Sugar customers have started running into an issue with emojis when using MySQL databases or the Sugar cloud. What could be so challenging about a poop emoji, you may ask? For that you may need a quick lesson on Unicode.

 

Character encoding and the beginnings of Unicode

Many of you are programmers, so you likely know that computers run entirely on (binary) numbers. As you type away on your keyboard, the stream of characters that gets stored in memory is a sequence of numbers based on the character encoding in use. "A" could be stored as 65, while "a" could be stored as 97, and so on. Programmers are obsessed with efficiency, so they would create many different character encodings for different alphabets or character sets to only use as little space as possible.

 

This became a real challenge as we started to share documents between systems via e-mail or the world wide web. Identifying and using the right character encoding became very important and also very difficult. If you didn't know the character encoding then you had to guess and if you guessed wrong then the text would not be displayed properly at all.

 

Encoding gone wrong. By Wikimedia Foundation (Wikimedia Foundation) [<a href="https://creativecommons.org/licenses/by-sa/3.0">CC BY-SA 3.0</a> or <a href="http://www.gnu.org/copyleft/fdl.html">GFDL</a>], <a href="https://commons.wikimedia.org/wiki/File:Mojibakevector.png">via Wikimedia Commons</a>

 

Fortunately, a group of companies, organizations, and individuals decided to come together to form the Unicode consortium. The Unicode consortium's primary mission has been to standardize character encoding into the Unicode standard that is widely used today. The most common Unicode character encoding is UTF-8 which is a variable length character encoding that can store any Unicode code point (which includes a growing number of emojis) into a number that is 1 to 4 bytes long.

 

MySQL's utf8 and utf8mb4 encodings

Programmers are obsessed with efficiency. Many programmers will not use 4 bytes if they think 3 bytes will do. When MySQL first implemented UTF-8 years ago, they decided that supporting the Basic Multilingual Plane (BMP) was good enough since it contained all the characters and most symbols used in modern languages. These characters require no more than 3 bytes. Adding support for the Supplementary Multilingual Plane (SMP) which includes emojis would mean that some characters would need to be stored in 4 bytes, which was just one byte too many for somebody.

 

Whatever the reason, MySQL's first implementation of UTF-8 that they called utf8 did not fully support UTF-8. When they fully implemented UTF-8 years later, they called it utf8mb4. This has been a source of confusion for many years since MySQL's utf8 character set seems to work fine until you run into certain special characters like emojis. Trying to store an emoji in MySQL's utf8 results in failure and the loss of data.

 

Sugar Summer '18 migrates from utf8 to utf8mb4

Sugar Summer '18 will use utf8mb4 for MySQL. This will allow users to import and display records and emails that contain emoji and other characters in the SMP. This feature will also be rolled into a future on-premise release.

 

The Sugar Summer '18 release will be available for Sugar cloud customers in a few short weeks.

 

Emojis in Sugar!

 

This character set and collation will be set automatically for new instances of Sugar and updated during the upgrade of existing instances of Sugar.  

 

If you have changed the default DB collation and you are in the Sugar cloud or on-premise using a MySQL database, then you should ensure that your collation is utf8mb4 compatible prior to upgrade. The collation can be set by the admin in Admin > Locale > Collation or by modifying config_override.php. If your collation is utf8mb4 compatible, the upgrade will automatically migrate the collation to utf8mb4. For example, if you have set your collation to be utf8_swedish_ci, the upgrade will migrate the collation to utf8mb4_swedish_ci. If no collation is set, Sugar will use the default utf8mb4_general_ci.

 

Database tables with very large row sizes (for example, custom tables with a large number of custom fields) may be unable to be automatically upgraded. The upgrader will notify you if a table would exceed the single-row size supported by MySQL (65,535 bytes) upon conversion to utf8mb4. In order to reduce the row size, we recommend the following:

  • Remove any/all fields that are not being used.
  • Reduce the lengths for char/varchar fields (e.g. size of longest existing value plus some padding).
  • Replace large varchar fields with text fields. Text fields are roughly 10 bytes, so significant size reduction exists when text fields can replace large varchar fields (e.g. VARCHAR(255).

For years SugarCRM has published database schema information for each new Sugar release at apidocs.sugarcrm.com. Generated documentation, like our database schema reference, has largely taken a back seat to more formal documentation like the Sugar Developer Guide and Sugar Developer training curriculum over at Sugar University.

 

Over the past couple of releases, we have improved the Sidecar framework code to have better structure and code documentation while preserving compatibility. One of the benefits has been a significant improvement in the quality of our generated Sidecar JavaScript documentation when using tools like JSDoc.

 

Anyone can now view the documentation at anytime for each new Sugar release without having to manually generate the documentation!

 

If you visit SugarCRM Generated Documentation, you will find the Sugar 8.0.0 Sidecar framework documentation available today. We will be publishing the generated Sidecar framework documentation for each new release in the future.

 

We plan to continue to invest in the quality and quantity of generated API references that we publish for each release.

Yes, you read that right!  We have not one but TWO ways for Sugar developers to earn a FREE pass to SugarCon 2018 in Las Vegas, Nevada.  

Speak at SugarCon!

SugarCon is evolving, and we want you to be part of it!  

 

We know that you, our Sugar Developers, are in the trenches everyday--solving customer problems, designing ways to make your jobs easier, and innovating.  You have expertise to share, and we want to hear it! We’ve created a list of suggested topics, but we encourage you to submit any talk that you would want to see at SugarCon.  Suggested topics include:

  • What you wish you knew about Sugar before you started
  • Best practices and war stories around Sugar customization or integrations
  • Tools you use to work with Sugar
  • Building a highly functional Sugar development team
  • Tips for working in Sugar Cloud

 

In addition to all of the amazing keynotes, networking opportunities, and lunchtime table topics at SugarCon, we are planning to have sessions specifically for developers.  This is where we want your expertise! You can expect that the audience at these developer focused sessions will be technical and have a copy of Sugar installed on their laptops with the Professor M’s School for Gifted Coders module loadable package and sample data installed.  Our goal is for these sessions to be hands-on or interactive. Bonus points if you leverage Professor M’s School for Gifted Coders as your example! Sessions will be 45 minutes long if we pair you with a SugarCRM co-speaker or 20 minutes long if you speak solo.

 

Our call for speakers is officially open at sugarcon.com.  Click the Tell Your Story button and tell us what you want to speak about.  In your submission’s description, be sure to indicate that this is a developer focused session.

 

We will be accepting submissions through July 13, 2018.  Availability is limited, so please make your submissions as compelling as possible.  

 

We’re excited to see your proposals!

 

Become a Sugar Scholar!

We’re excited to be introducing the Sugar Scholars program.  Sugar Scholars are people who are advocates for Sugar all year round and want to help make SugarCon as amazing as possible.

 

You might be a Sugar Scholar if…

  • You are an active member of our Developer Community.
  • You rack up tons of points in Sugar Insiders.
  • You frequently blog about Sugar.
  • You create engaging videos about Sugar.
  • You can’t stop Tweeting about Sugar.
  • You lead Sugar meetups.

 

Sugar Scholars will be involved in assisting with SugarCon on both days of the conference.  They may be asked to do things like the following:

  • Facilitate lunchtime table topics for developers
  • Greet developers at the developer-focused lunches, sessions, and happy hours
  • Give a 5-minute lightning talk

 

If you think you have what it takes to be a Sugar Scholar, fill out this form by July 13, 2018.

We've already begun preparations for SugarCon in fabulous Las Vegas, Nevada, and we REALLY want to see YOU there.  That's why we have a promotion just for you, our favorite developers! 

 

From now until June 30, developers can register for SugarCon for only $400 when someone else from their company registers at the published rate for the Full Conference or VIP Full Conference (limit one discounted developer registration per Full Conference or VIP Full Conference registration).  To take advantage of the discount for developers, your colleague should register for SugarCon at the published rate.  Then email us at sugarcon@sugarcrm.com with your colleague's name and indicate you'd like the promo code for developers.  We'll reply with the promo code you need in order to register.

 

To make this deal even sweeter, Sugar Insiders can currently get our early-early bird rates for the Full Conference through June 30.  Not yet a Sugar Insider?  No problem!  Sign up here!  Once you're a Sugar Insider, email sugarcon@sugarcrm.com to extend our early-early bird rate of $600 per registrant until June 30, 2018.

 

Need help convincing your boss to bring you?  We've written this handy letter you can customize and send to your boss:

 

Hello, [your boss’s name] - 
I’d like to attend SugarCon 2018 on October 10-11, 2018 in Las Vegas, NV. SugarCon is the annual conference for Sugar users, experts, SugarCRM staff, thought leaders, and developers. 
I’d like to attend SugarCon for several reasons. There are multiple ways that my attendance can benefit [your organization’s name].
These include:
I’ll be a producer for the team - In addition to hands-on training, there will opportunities to engage with new tools and techniques in order to improve our current implementation.
I’ll learn more about CRM - SugarCon has sessions on challenges+solutions, sessions on practical pieces of the system as well as sessions on possible future projects. There will be panel sessions as well on how other brands are using the system to create real business outcomes.
I’ll connect with and learn from my networks - SugarCon will have people across various industries, roles, company sizes, and ages of deployment. I’ll be able to exchange ideas within these various groups in order to solve our unique business challenges. Plus, there will be networking events and sessions just for developers, so I can connect with other people tackling the same challenges I am.
The investment is approximately $XXXX.XX, which includes the discounted registration fee, air travel, hotel nights at the Cosmopolitan under the group block, and transportation between the airport and hotel.
As a developer, if I attend with someone else from our company, I can take advantage of a discounted registration fee just for developers and the investment for me goes down to $XXXX.XX.
I look forward to growing professionally and strengthening my network at SugarCon!  Thank you for your consideration!

 

We can't wait to see you in Las Vegas in October!

Hey there, developers!  We've officially released Sugar 8.0.0 (Spring '18)!  This is our third quarterly release for Sugar Cloud (formerly known as Sugar On-Demand).  We're on a roll!  This release is also our yearly release for our on-prem customers.

 

This release has a ton of great features, but we're most excited about all of the work we put in to data privacy around the General Data Protection Regulation (GDPR).  If GDPR is just a random smattering of letters to you, check out Matt's blog posts on the subject.

 

Our Co-Founder & CMO, Clint Oram, discusses the highlights of this release from an end-user's perspective in the video below:

 

 

Matt Marum and I recently hosted a webinar where we gave an overview of the big things developers need to know about this release:

 

 

The slides from the webinar are available here.

 

If you're looking for the high-level overview, I've got you covered!

  1. In order to support the GDPR and other data privacy regulations, we added a Data Privacy module and a Data Privacy Manager role.  We added additional features like the ability to opt-out new email addresses by default, mark fields as containing personally identifiable information (PII), disable activity streams, track additional context in audit logs, and view and permanently erase PII.  For more information, see Data Privacy and GDPR in Sugar.
  2. We made updates to the audit data model to better support tracking information about changes.
  3. We implemented semantic versioning for our REST API and introduced v11.1.
  4. We added a new option to configure API platforms in the Admin interface.  Integrations that use custom platforms no longer need to create a custom module loadable package in order to register their custom platforms; the platforms can be registered in the Admin interface instead.
  5. To improve performance, ./cache/file_map.php has been removed and many SugarAutoloader functions have been deprecated and now do nothing. We also recommend disabling the 'block_nested_loop' optimization in MySQL.
  6. Be sure to check the updated list of supported platforms.  Note that we have dropped support for PHP 5.6 and Elasticsearch 1.7.5.
  7. We will not be posting this release in the Developer Builds community since it will be available in the Sugar Store.

 

This blog post promised to have just about everything you need to know about the Sugar 8.0.0 (Spring '18) release.  Below are some resources that have the rest of the details.

 

We hope you’re as excited about this release as we are!

Sugar Summer '18 introduced confirmation links to allow customers to gather affirmative consent for using email addresses in campaigns. Check out Sugar Summer '18 (8.1.0) Overview for Sugar Developers for more details.

 

As discussed in Why SugarCRM developers should care about Data Privacy and GDPR, data privacy is an important topic that we are going to keep discussing here on the developer blog. Today is another installment specifically focused on best practices for getting consent to collect and process personal information.

 

General Data Protection Regulation (GDPR) has strict requirements for consent. However, consent to personal data processing is a big topic for regulation in many countries and especially applies to email marketing. Under GDPR, unless you have an existing lawful basis for using personal information, you need to collect explicit consent from the data subject when storing and processing their information in Sugar. By explicit consent, we mean it was freely given, informed, affirmative, and unambiguous. For example, a web to lead form with a pre-selected opt-in checkbox is not going to cut it in the EU, Australia, or Canada.

 

Let's look at a couple different ways of collecting explicit consent.

 

Collecting explicit consent

 

Collecting explicit consent is particularly important for marketing campaigns and business development. For other CRM use cases (like customer support), you may already have another lawful basis that gives you the right to process customer information.

 

Consent really just boils down to allowing someone to knowingly and freely give their permission for you to use their personal information. You could collect consent over the phone, written in an email, or over a handshake in person.

 

It is possible to collect explicit consent with a properly designed web form as well. There are lots of resources out on the web with advice for building compliant Single Opt-In (SOI) web forms.

 

However, a common practice for collecting consent is to use Double Opt-In (DOI) which is also known as Confirmed Opt-In (COI).

 

Double Opt-In (aka Confirmed Opt-In)

 

We all know web forms aren’t perfect. Sometimes users aren’t sure what they are signing up for themselves. Maybe they mistyped their email address and now somebody else is signed up for that mailing list. Or worse, somebody is a troll and intentionally signing them up for email or services that they don’t want. There is a better way and it is called double opt-in.

 

Double opt-in (or Confirmed Opt-In) essentially means that someone who puts their personal information into a web form must confirm twice. A user enters their information on a form and affirms that they want to subscribe to a service. They then get a confirmation email, text message, or some other communication where they can re-confirm their intention. This is typically done by providing a link in an email that users must then click to confirm.

 

Double opt-in is an easy way to ensure you have a clean mailing list or lead list full of people who went through the trouble to confirm twice.

 

Many of our customers use marketing tools such Act-On, Marketo, and others to manage their mailing lists and the necessary opt-in processes. As an added feature to Sugar’s Marketing capabilities, we are planning to include DOI/COI in a subsequent release. Meanwhile, here is how you can achieve DOI/COI with Sugar 8 or Spring ‘18.

 

Adding Double Opt-In to Sugar 8.0.0

 

If you have access to the Sugar 8 Preview then this is something you can try today. Otherwise you will need to wait until the official release in a couple short weeks.

 

This customization implements double opt-in behavior for specific email addresses stored within Sugar. The steps below involve Sugar Studio configuration changes as well as code level customizations. This example allows for any email address to be subject of a confirmed opt-in flow without any specific dependencies on Campaigns, web to lead forms, Leads, Contacts, etc. Note that since this customization is focused on the confirmation step of double opt-in so for this reason we prefer the term “Confirmed Opt-In”.

 

Once you have followed the steps below, you will be able to generate confirmation messages for new email addresses. For each email address that needs to be confirmed, there will be an associated URL that, when clicked, will confirm and opt-in the associated email address.

 

Step 1) Configure your lead forms or integrations to opt-out by default

 

You need to take some steps to make sure that email addresses are not prematurely opted-in within Sugar.

 

In Sugar administration, visit the “System Email Settings” panel and make sure “Opt-out new email addresses by default” is checked. You may need to enter SMTP Mail Server settings in order to save this change on this page. This is a new setting in Sugar 8.0.0.

 

 

If you are using Sugar web-to-lead forms, they should be updated to remove the email opt-in checkbox that Sugar includes by default if it still exists. I’d recommend updating the form to let the user know that a confirmation email will be sent separately.

 

 

If you are using some other system (like a lead portal or marketing automation tool) to capture contact information, they should be configured to not send back email opt-out information to Sugar. Or alternatively they could be configured to set email opt-out to true by default.

 

Step 2) Add Studio configuration changes to Data Privacy module

 

In this example, we will reuse the Data Privacy module (new in Sugar 8.0.0) to track email confirmation activities. If you haven’t already, you may need to add the Data Privacy module to the list of displayed modules in the Administration panel first.

 

Add a new “Confirmed Opt In” option to the dataprivacy_type_dom used by the Data Privacy module’s type field. This will be used to differentiate email confirmations from other Data Privacy activities.

 

 

Also add two new custom fields that are visible only for Confirmed Opt-In activities. To do this, I used the following visibility formula on these custom fields.

 

equal($type, "Confirmed Opt In")

 

The first is an email_address_to_confirm_c text field that stores the record ID for the email address that needs to be confirmed.

 

 

The second is a URL field called confirmation_url_c that will use a generated URL. The default value is designed to pass email_address_to_confirm_c field and the Data Privacy id field as parameters to a custom entry point that we will create later. The URL will vary based upon the site URL for your Sugar instance. For my development environment, I used:

http://localhost:8080/sugar/index.php?entryPoint=confirmEmail&email={email_address_to_confirm_c}&id={id}

 

 

Make sure to add these two new fields to your Data Privacy record layout as well which will help us with testing. Remember to Save and Deploy after adding them.

 

 

Step 3) Add the ConfirmedOptInManager class and related Entry Point

 

The following class includes two optional logic hooks as well as a required method that will be used by a new Entry Point. We are choosing to use a custom Entry Point here instead of a REST API endpoint because the Sugar REST API is designed to only return JSON responses. This makes for a poor usability experience for the person who is trying to confirm their email address and sees a page full of gobbledygook in their browser. Typically, people will expect to see a nicely formatted web page with a confirmation message. An entry point allows us to return an HTML response that matches user expectations.

 

The createConfirmedOptIn() method can be used as an after_save hook on the Leads module to automatically create a confirmed opt-in activity whenever new Lead records are created.

 

The sendConfirmOptInMessage() method is a stub that can be used as an after_save hook on the Data Privacy module to automatically send confirmation emails to end users. This could also be orchestrated using an Advanced Workflow.

 

The last confirm() method is the most important one. This method is called directly by a custom public entry point shown further below. This is what determines if the request is a valid one and then updates the email address record. It first loads the DataPrivacy bean, then compares the email address ID stored there with the one provided in the URL. It then updates the EmailAddress and DataPrivacy beans accordingly.

 

./custom/Extension/modules/Leads/Ext/LogicHooks/createConfirmOptInRecord.php

 

<?php
// Copyright 2018 SugarCRM Inc.  Licensed by SugarCRM under the Apache 2.0 license.
$hook_array['after_save'][] = array(
  1,
  'create_confirmation_record',
  null,
'Sugarcrm\\Sugarcrm\\custom\\ConfirmedOptIn\\ConfirmedOptInManager',
  'createConfirmOptInRecord',
);

 

./custom/Extension/modules/DataPrivacy/Ext/LogicHooks/sendConfirmOptInMessage.php

 

<?php
// Copyright 2018 SugarCRM Inc.  Licensed by SugarCRM under the Apache 2.0 license.
$hook_array['after_save'][] = array(
  1,
  'send_confirmation_message',
  null,
'Sugarcrm\\Sugarcrm\\custom\\ConfirmedOptIn\\ConfirmedOptInManager',
  'sendConfirmOptInMessage',
);

 

./custom/src/ConfirmedOptIn/ConfirmedOptInManager.php

<?php
// Copyright 2018 SugarCRM Inc.  Licensed by SugarCRM under the Apache 2.0 license.
namespace Sugarcrm\Sugarcrm\custom\ConfirmedOptIn;

/**
*
* Manages confirmed opt-in workflow for Email addresses
*
* Class ConfirmedOptInManager
*/

class ConfirmedOptInManager
{

    /**
     * After Save logic hook on Leads module (or any other Person module with an email address)
     *
     * Used to automatically create email confirmation records in DataPrivacy module.
     *
     * @param \SugarBean $bean SugarBean with an email address that needs to be confirmed.
     * @param string $event Logic hook event name. Expected to be 'after_save'.
     * @param array $arguments
     */

    public function createConfirmOptInRecord(\SugarBean $bean, string $event, array $arguments)
    {
        //If this is a newly created record
        if (!$arguments['isUpdate'] && $bean->load_relationship('emails')) {
            $address = $bean->emailAddress->getPrimaryAddress($bean);
            //And a primary email address is set
            if (!empty($address)) {
                //Create a new 'Confirmed Opt In' DataPrivacy activity
                $dpb = \BeanFactory::newBean('DataPrivacy');
                $dpb->name = "Confirm $address";
                //Note: This assigned user for Data Privacy activity MUST have the Data Privacy Manager role!
                $dpb->assigned_user_id = $bean->assigned_user_id;
                //We will store ID for primary address here.
                $dpb->email_address_to_confirm_c = (new \SugarEmailAddress())->getGuid($address);
                $dpb->type = 'Confirmed Opt In';
                $dpb->save();
            }
        }
    }


    /**
     *
     * After save logic hook on Data Privacy module
     *
     * Can be modified to send the confirmation message to the appropriate email address.
     * You should make sure not to send multiple messages for the same Data Privacy request.
     *
     * @param \DataPrivacy $bean DataPrivacy bean that has been saved
     * @param string $event Logic hook event name. Expected to be 'after_save'.
     * @param array $arguments
     */

    public function sendConfirmOptInMessage(\DataPrivacy $bean, string $event, array $arguments)
    {
        //Should only continue if this is a newly created Confirmed Opt In request
        if ($bean->type === 'Confirmed Opt In' && !$arguments['isUpdate']) {
            $log = \LoggerManager::getLogger();
            $eab = \BeanFactory::retrieveBean('EmailAddresses', $bean->email_address_to_confirm_c);
            if (!empty($bean->email_address_to_confirm_c) && !empty($eab->email_address)) {
                $log->info(
                    "Send out confirmation to $eab->email_address with link to $bean->confirmation_url_c."
                );
                // You could update to send message using mailer, SMS gateway, whatever.
            }
        }
    }

    /**
     *
     * Opt-in the email address if the email associated with given requestID matches the given emailAddressID
     *
     * Used with `confirmEmail` public custom entry point
     *
     * @param string $requestId
     * @param string $emailAddressId
     * @return bool
     */

    public function confirm(string $requestId, string $emailAddressId): bool
    {
        global $current_user, $timedate;
        // Since this is a public endpoint, we need to select a current user to associate with this action.
        // For our purposes, we will use the User assigned to the Data Privacy request since the person confirming
        // their email address is likely not a current Sugar user.
        // Note: For this to work, assigned/current user must be admin or have Data Privacy Manager role!
        if ($current_user && empty($current_user->id)) {
            $q = new \SugarQuery();
            $q->from(\BeanFactory::newBean('DataPrivacy'), array('team_security' => false));
            $q->where()->equals('id', $requestId);
            $q->select(array('assigned_user_id'));
            $results = $q->execute();
            //If no assigned user, then default Admin user will be used.
            $assignedUserId = empty($results) ? 1 : $results[0]['assigned_user_id'];
            $current_user = \BeanFactory::retrieveBean('Users', $assignedUserId);
        }

        $confirmRequest = \BeanFactory::retrieveBean('DataPrivacy', $requestId);
        //Make sure email address matches the one associated with the request ID
        if ($confirmRequest && $confirmRequest->email_address_to_confirm_c === $emailAddressId) {
            //If already closed, then no need to re-confirm.
            if ($confirmRequest->status === 'Closed') {
                return true;
            }
            $addrBean = \BeanFactory::retrieveBean('EmailAddresses', $emailAddressId);
            $addrBean->opt_out = false; //opt in!
            $addrBean->save();
            $confirmRequest->status = 'Closed';
            $currentTime = $timedate->getNow();
            $confirmRequest->resolution = 'Confirmation received on ' . $currentTime->format(DATE_RFC850);
            $confirmRequest->date_closed = $currentTime->asDbDate();
            $confirmRequest->save();
            return true;
        }
        return false;
    }

}

 

The last file is our public custom entry point, which we install with the help of the EntryPointRegistry extension. This will allow us to create a publicly accessible URL on our Sugar instance that can be used to easily display HTML for a user readable response.

 

./custom/Extension/application/Ext/EntryPointRegistry/confirmEmail.php

 

<?php
// Copyright 2018 SugarCRM Inc.  Licensed by SugarCRM under the Apache 2.0 license.
$entry_point_registry['confirmEmail'] = array(
'file' => 'custom/src/ConfirmedOptIn/ConfirmEmailEntrypoint.php',
'auth' => false
);

 

Since this is a public entry point, we should take extra care to validate user input. We will use the InputValidation framework to assert that the URL parameters are both the GUIDs that we expect for record IDs.

 

./custom/src/ConfirmedOptIn/ConfirmEmailEntrypoint.php

 

<?php
// Copyright 2018 SugarCRM Inc.  Licensed by SugarCRM under the Apache 2.0 license.
if(!defined('sugarEntry') || !sugarEntry) die('Not A Valid Entry Point');

use Sugarcrm\Sugarcrm\Security\InputValidation\InputValidation;
use Sugarcrm\Sugarcrm\custom\ConfirmedOptIn\ConfirmedOptInManager;

$manager = new ConfirmedOptInManager();
$validator = InputValidation::getService();

$recordId = $validator->getValidInputGet('id', 'Assert\Guid');
$emailId = $validator->getValidInputGet('email', 'Assert\Guid');

if ($manager->confirm($recordId, $emailId)) {
    echo "<h1>Confirmed!</h1>";
} else {
    echo "<h1>Unable to confirm.</h1>";
}

 

This page will just show “Confirmed!” if the email address was properly confirmed and “Unable to confirm.” if the confirmation failed which could be due to badly formed request or because the email address or data privacy record IDs didn’t already exist in the system. You could update this entrypoint to return whatever HTML you want or perhaps even to redirect the person’s browser to some other page.

 

For example,

 

header("Location: http://www.example.com/success");

 

If you happen to see a blank page, then input validation has probably failed. You can check for a fatal message in sugarcrm.log for more details on the failure.

 

Step 4) Try it out!

 

Create a new Lead record that includes a new email address.

 

New Lead record

 

Switch over to Data Privacy module and you will find a new Data Privacy record like the one below.

 

 

For convenience, we can just click the confirmation URL itself to trigger the confirmation check of using an email to do it. In a real implementation, it may make sense to hide this URL from Sugar end users so they do not accidentally click on it since we expect someone who does not use Sugar to click it themselves.

 

 

Nice, it was confirmed! If the request isn’t quite right or the email address is not in the system then you will get the “Unable to confirm” message instead.

 

If I return to my Data Privacy record, then I can see it has been completed and that the email address on my original Lead record has been opted in.

 

 

There is a lot that could or should be done to enhance this example for production use. But you can see that the pattern is straightforward to implement using existing Sugar platform extensions and APIs.

While we introduced PHP 7.1 support in the Sugar 7.9.0 release back in May 2017, we have up until now continued to support PHP 5.6. But PHP 5’s days are numbered. We will be dropping support for PHP 5.6 in the Sugar 8 and Spring ‘18 releases. Sugar 8.0.0 and Spring ‘18 will only support PHP 7.1. Long live PHP 7!

 

In the Sugar cloud, we have been migrating from PHP 5.6 to PHP 7.1. With this transition, we have discovered some additional PHP 7 pitfalls that we want to make sure everyone avoids in their Sugar customizations. These problems affect any version of Sugar that uses PHP 7. They are both centered around a new ArgumentCountError that was introduced in PHP 7.1.

 

If you haven’t already, you should check out the old blog post we did on preparing customizations for PHP 7. There are lots of benefits for using PHP 7 with Sugar. Not only are there performance improvements but Sugar code as well as customizations are now able to take advantage of more advanced PHP language features.

 

Let’s take a look at the two pitfalls we want to help you avoid.

 

Do not use deprecated PHP 4 style constructors in Logic Hooks classes

You should not be using PHP 4 style constructors in PHP 7 at all, but there is an additional complication that is introduced when you are using Logic Hooks with PHP 7.1.

 

Take the following example:

 

class Hook {
    public function hook($bean, $event, $arguments)
    {

    }
}

 

The issue here is rooted in the fact that functions and classes in PHP are case insensitive.  The hook() function contained within the Hook class is interpreted as a PHP 4 style constructor even though the names differ in case.

 

For logic hook code where the class name and function name differ only in case, it would cause Sugar to throw fatal PHP errors in PHP 7.1 and warnings in earlier PHP versions. Since the class name and function name does not match in the example, Sugar will not recognize hook() as a constructor. So Sugar will first instantiate the Hook object and then subsequently call the hook() function later. But PHP recognizes hook() as a constructor. So PHP would instantiate a Hook object without passing the needed arguments to hook(). Calling a function with an incorrect number of arguments in PHP 7.x throws a fatal ArgumentCountError.

 

When using PHP 5 with Sugar, this would generate a PHP warning and ultimately the hook() function would be called twice; once with no arguments for the constructor and then a second time with arguments to process the hook.

 

So there’s all kinds of things going wrong here in this very brief example.

 

For compatibility reasons, we have fixed this case sensitive behavior where we will only call this function once in Sugar 8.0.0 but even then you are still using a deprecated PHP 4 constructor in PHP 7. This is either unintentional or simply a bad idea.

 

You must use function names that are different than your class names in your Logic Hook classes.

 

We can easily fix the example above by changing the function name.

 

class Hook {
    public function afterSave($bean, $event, $arguments)
    {

    }
}

 

 

Do not define too many arguments for Logic Hooks functions

Now we should take a look at another issue that will throw an ArgumentCountError.

 

class Hook {
    public function afterSave($bean, $event, $arguments, $customArgs)
    {

    }
}

 

The developer who wrote the afterSave() function above may have been trying to save time by adding the $customArgs argument to a standard after_save module logic hook function to handle some alternative usages. The issue is that Sugar is going to call this logic hook function with three arguments instead of four, which will throw an ArgumentCountError in PHP 7.1. In PHP 5, a warning is simply logged and the function is still executed.

 

It should go without saying that functions should be called with the correct number of arguments.

 

¯\_(ツ)_/¯

 

We can fix the example above to ensure that functions can always be called with the correct number of arguments.

 

class Hook {

    public function afterSave($bean, $event, $arguments)
    {
        $this->otherSave($bean, $event, $arguments, array());
    }

    public function otherSave($bean, $event, $arguments, $customArgs)
    {

    }

}

 

You must ensure that your Logic Hook functions use the correct number of arguments.

 

For most logic hook events that is three arguments but sometimes it is two! Check the event definitions in the Logic Hooks section of the Developer Guide if you aren’t sure how many arguments are required!

 

Recap

Let’s recap what you should do to prepare your Logic Hooks for PHP 7.1:

  • Ensure that none of your Logic Hook classes contain functions have the same name as the class. Remember that PHP is case insensitive when it comes to class and function names.
  • Ensure that your Logic Hook functions use the correct number of arguments.

Hi there, Sugar Developers!

 

We want to make sure your customizations and integrations are ready for the Sugar 8 and Spring ‘18 releases, so we’re hosting two webinars just for you!

 

Webinar Topics

We’ll discuss the big changes that are likely to impact you including the following:

  • Review Data privacy features used for GDPR compliance (if you’re not thinking about this, you probably should be!)
  • New Sidecar components and features that support Data Privacy
  • New Admin panel to register custom API platforms
  • Changes to Audit APIs, data model, and behavior
  • Changes to supported platforms and libraries
  • Update on the Professor M School

 

Webinar Information:

We are holding 2 sessions to accommodate various geographical locations. Please choose ONE of the following times below.

Monday, April 23nd 6:00 PM - 7:00 PM PT

OR

Tuesday, April 24rd 8:00 AM - 9:00 AM PT

(Choose one)

 

Register Here


We will be posting the webinar recordings here in the Developer space for those who are unable to attend the live sessions.

Post originally written by avlasovsugar.

 

Alex Vlasov is an experienced Sugar Developer and contractor that works on the SugarCRM Engineering platform team. In this post, Alex shares advice to Sugar Developers for preparing their Sugar customizations for PHP 5.6 and PHP 7.

 

The Sugar team has been preparing core Sugar code for PHP 5.6 and PHP 7 support with an eye on deprecating support for older PHP versions. This means there are some actions that the Sugar Developer community needs to take in order to prepare their code for latest versions of PHP.

 

Summary of core Sugar app changes

 

In order to support newer versions of PHP, we made some changes to internal core classes that Sugar Developers need to know about. Many of these changes were made in Sugar 7.7.1 when we added PHP 5.6 support. Other changes outlined below are in upcoming Sugar releases.

 

PHP 7 deprecated features removed from core Sugar code

  1. Removed static calls to non-static methods
  2. Removed PHP4-style constructors

 

Read the full list of features deprecated in PHP7.

 

Additional changes to address core Sugar code hygiene and to adopt new PHP 7 features

  1. Fixed incompatible child class methods signatures
  2. Updated 3rd party libraries to support latest versions of PHP
  3. Changes to support PHP 7.0 new uniform variable syntax
  4. Adopted PHP 7's CSPRNG API but added random_compat library to support older PHP versions

 

Actions for Sugar Developers

 

In order to properly upgrade custom code, two main tasks need to be performed:

 

Make your code compatible with PHP 5.6 and PHP 7.0

 

To make your code compatible with PHP 5.6 and 7.0, use the following checklist.

  1. Remove static calls to non-static methods within your PHP code
  2. Remove any use of PHP4-style constructors within your PHP code
  3. Remove other deprecated PHP functionality in your PHP code (PHP 5.6 deprecated features, PHP 7 deprecated features)

 

For example,

 

class Foo {

public function __construct() { // Yes.
// ...
}

public function Foo() { // No.
// ...
}

    public static function aStaticMethod() {
        // ...
    }

    public function aNormalMethod() {
    // ...
    }

}

Foo::aStaticMethod(); // Yes.

Foo::aNormalMethod(); // No.

 

 

 

For additional detail, refer to PHP migration guides for PHP 5.6 and PHP 7.0.

 

Make your code compatible with changes to Sugar PHP APIs

 

To make custom code compatible with updated Sugar PHP APIs, use the following checklist.

  1. (7.7.1) Do not use old PHP4 constructors in Sugar PHP classes, they are removed as part of PHP 7.0 support.
  2. (7.7.1) If there is code that extends Sugar PHP classes, make sure child methods signatures are correct
  3. (7.7.1) 3rd party libraries were moved:
    1. parsecsv library was moved to the vendor directory
  4. (future) 3rd party libraries updates:
    1. Elastica library will be upgraded, so make sure you do not use deprecated Elastica APIs.
    2. Removal of mcrypt support and Crypt_Blowfish library, so make sure you are not calling these libraries directly.
  5. (future) Significant changes to be made to the following Sugar PHP APIs:
    1. ActivitiesApi::getQueryObject() will become protected and function arguments are changed
    2. RelationshipHandler class will stop extending Relationship class
    3. SearchForm class will not extend EditView class anymore
    4. Quarter*TimePeriod::buildLeaves() methods will have function arguments changed
    5. PMSEEngineFilterApi::addFieldFilter() method will be renamed