Forums - Thread Details
  • State Verified Answer
  • Replies 11 replies
  • Subscribers 319 subscribers
  • Views 2261 views
  • Users 0 members are here
Related

Workflow to ship through Git a custom module without having every developer/server to manually load it

Jeff Girard

Hello!

(Sorry for the length but I think it could help many people, so I will take the time to explain)

My teammates and I are working on some custom developments into SugarCRM (version 7.6), and we have some trouble to understand what is the right way to work with custom module versus git.

Note that we are totally controlling all the stack from dev to production, we are hosting on-site so the application and infrastructure will be under our control. We have:

- 1 local development server in our office

- 1 aws test server

- 1 aws production server

- Each developer has it's own local setup

- We are using git, and are used to since many years (maybe its why we are getting difficulties understand the sugarcrm-gitflow, its quite unclear).

All of these environments ^ has its own database also. Nothing shared.

The problem we have: We are developping custom modules , and we want to ship these module through git only, without having to package the modules and use the "module loader" on each environment after to install them.

The reasons why we want to do this is that it doesnt make sense to us to follow a git process which implies manual action on every environment that could get our working directory dirty after (for example) a module loaded. And also, when to load a module, when to pull, in which order? You see, its very weird no? The only things that surely we have to manually deal with it is regarding the SQL for the tables "fields_meta_data" and "relationships", actually we will ship the sql changes through a custom "sql migration" directory (experimental ).

Maybe its not possible, but our best process would be: develop, commit+push, pull on environment x, fix permission, repair & rebuild, maybe run some sql => enjoy!

If possible, I just want to know how to achieve this, what we need to do under the custom folder to have our module pre-installed in the codebase.

If not possible, what could we do? The process need to be clear for custom module development with multi-enviroment, and must imply git. (And please, I don't want to get a dirty working dir after using module loader or repair/rebuild, this doesnt make sense when deploying to a server... gitignore is the solution maybe?)

*By the way, we want to be sure that we are not screwing up our SugarCRM installation, upgrade need to be possible without lot of pain, and we want to follow the best practices!

Thanks a lot for any help!

P.S. I've googled a lot since two weeks to found answer to our problem, I found some partial answers, some interesting articles, but nothing that really answer to our question)

  • 0

    The process that you've described is substantially similar to processes I've seen most Sugar Partners adopt.

    If you have all the custom module code and extensions in place then a Quick Repair and Rebuild will be sufficient to deploy your custom modules. Setting up your .gitignore appropriately is important.  Check out this recent thread. Re: Best Practices for version control in v7.x?

    We definitely recognize the importance of providing better guidance and enablement for Sugar project deployment.  This was a topic of discussion at Tech Track for our recent Global Partner Summit.

    App Ecosystem @ SugarCRM

  • 0 in reply to Matt Marum

    Thanks Matt! ok so I'm happy that we are not completely wrong with our process. And regarding the .gitignore file, and based with the process I described in my previous post, what should be it content?

    I've found this template edited in 2013 (maybe its outdated no?):

    gitignore/SugarCRM.gitignore at master · github/gitignore · GitHub

    I want to be sure that this won't break our custom development... The files modified under /custom/modulebuilder can be really be excluded, it is safe?

    Once again, I've found some posts about the gitignore, always different from one to another, it should be standard for a SugarCRM version I think... (example for 7.6, everyone would have mostly the same gitignore file no?)

    I just want to implement one that I can trust, I don't want to go into a error/retry cycle or assume that it will be ok. Actually here it's something that block our developers (this make us going a bit insane hehe), the repair/rebuild step is messing up our git working directory so a good gitignore file will surely fix our worries!

    Thanks for any help regarding the .gitignore file, we want the best one!

  • 0 in reply to Jeff Girard

    Anyone can help me with this please?

    *For now we use the following .gitignore content provided by one of our SugarCRM partner, looks like it does the job, but surely in the community here there are some people who are aware of the best gitignore I should be using for the version 7.6...

    *.log

    /cache/

    /config.php

    /config_override.php

    /stats/

    /error/

    /.htaccess

    #the content of this folder is always rebuilt by sugar

    /custom/application/

    #the content of this folder is usually the changes e.g. when you update a recordview do a deployment this should be left out

    /custom/history

    # we don't need this file

    /custom/index.html

    #THIS IS IMPORTANT, we do not commit the content of the Ext folder since during rebuild and repair this gets overwritten.

    /custom/modules/*/Ext

  • 0 in reply to Jeff Girard

    Under the custom/ directory, the content of custom/application/Ext (global extensions) and custom/module/MODULE_NAME/Ext (module extensions) are generated doing Quick Repair.  Adding this to .gitignore is fine so long as you understand that running quick repair when you switch branches is necessary to overwrite any stale or missing extensions.

    That .gitignore file you found from 2013 is still pretty solid to me. Only thing I'd possibly add to it is config_si.php which is a silent install config file. Sometimes this can include passwords, etc, that you don't want checked into Git.

    Not much has changed about the basic structure of the back end of the application since Sugar 6.5.

    App Ecosystem @ SugarCRM

  • 0 in reply to Matt Marum

    Hi,

    I'm on the same path but we want to integrate Jenkins to the Dev Server so it will auto repair and rebuild, clear cache, give permisions and execute queries result from the reparation.

    You gave me a very clear idea to integrate Git.

    Also don't forget about this - Shane Dowling – SugarCRM - Git Version Control Strategy

    You didn't mention anything about the DB

  • 0 in reply to Gerardo Garcia Lima

    Hi,

    regarding the database, here is what we put in place:

    1. We created the folder "custom/sql"
    2. If a developper changes the content of the "field_meta_data" table, he has to create a sql file (migration pattern) where we see his changes - we deal with dashboards tables manually (without git) because its not something done by the developers, for the relationships table its not a problem since it is recreated upon repair/rebuild step.
    3. Push the code to branch x
    4. When other developers/servers pull the code from branch x, for the moment (we are no ready to automate this yet), someone is responsable of manually checking for any new sql to commit, and apply the change against the database environment (env = developer machine or a server x)

    It's surely not perfect, but since nothing is documented about the database changes in the SugarCRM documentation, we decide to implement our own process based on our deployment experience here (team of ~14 developers, we can't afford to develop through SugarCRM without any process).

    We are going live with SugarCRM in 1 month. You Gerardo Garcia Lima are you actually in production with SugarCRM? Are you or planning-to load balance the application on multiplce instance? (its what we are actually doing here on AWS)

  • 0 in reply to Jeff Girard

    Hi Jeff,

    Sorry, I didn't understand clearly the last paragraph, I'm a spanish speaker, so please if i answer something different to what you wanted to know please tell me

    We are actually developing custom functionality to SugarCRM, almost 5 months but we are 5 developers and no one has tried SugarCRM before so it was difficult at the beginning.

    Actually every developer has 1 Sugar instance on an Azure running RedHat. We integrate everything manually to a physical development server with more less the same config like the Azures. It takes lots of time, we don't use git or svn. Thats why I'm planning to implement Git, Almost done, and Jenkins. I haven't seen any documentation either.

    Actually everything you wrote it's very useful to me, thank you very much

  • 0 in reply to Gerardo Garcia Lima

    I'm glad I helped!

    (For the last paragraph, I was talking about the deployment in production but it's kind of out-of-scope for the current discussion, I will end it here to prevent mixing up two differents subjects inside one thread  )

  • 0 in reply to Jeff Girard

    Hi Matt,

    Thanks for your help on this.

    I'm trying to firm up a development workflow for a SugarCRM project I've recently joined.

    Would it be possible to also add compiled workflow and JS files to .gitignore? Something like this:

    src/main/php/custom/modules/*/workflow/workflow.php src/main/php/custom/workflow/plugins/plugin_list.php  *.js !src/main/php/custom/**/*.js !src/main/php/jssource/**/*.js

    Is this something you've encountered before?