Showing posts with label apex. Show all posts

Monday, May 30, 2011

[Salesforce] Scheduling an Apex call from the command line

1) Problem

This post presents a way to execute an Apex method or an Apex batch from the command line, regardless your platform (gnu/Linux, Unix, Windows) using Perl and cURL.

User Cases

Customer IT rules compliance: all the scheduled batches should be centralized.
Integration: customer needs to execute an Apex batch (e.g. after a daily data load.)
Customer IT does not have resources with the required skills to write and maintain an application that will deal with Salesforce API.

2) Solution big picture

This is a very simple solution that might be easily integrated in a shell script. The advantages are: no sweat for customer IT and the new capability of launching heavy processes running on the Salesforce.com side.

Technical solution in a nutshell

A shell script is calling a cURL command that deals with the Salesforce.com API. cURL posts a SOAP login message to Salesforce and opens a session. Then, cURL is calling your Apex method exposed as a web service.

The user case we will walk through

Let's assume that your customer has a nightly batch that imports new leads (with an ETL e.g.). But the phone numbers format does not meet his CTI requirements; so the system cannot match the inbound calls. We need to apply a filter on the phone numbers. The process stands in a simple string replacement using a regular expression: we will suppress all the none numerical characters with the pattern [^\d]*.

3) Solution deep dive

Why Perl?

Why Perl rather than Ruby or PHP? Perl is a popular interpreted script language that is usually pre installed on any Unix/Linux systems. Perl provides powerful text processing facilities without the arbitrary data length limits, facilitating easy manipulation of text files. (see Perl on Wikipedia )

Which Perl for your system?

Unix/Linux: pre installed (most of the times)
Windows: Strawberry Perl (open source), Active State Perl (commercial with a free community edition)

on Windows, the installation is straight forward and should not upset the system administrator. Choose your flavor: Strawberry or Active State.

Why cURL?

cURL is open source and stands for “Client for URLs”. It is a command line tool for transferring data with URL syntax, supporting HTTP, HTTPS, FTP, SMTP, LDAP, etc. cURL handles cookies, HTTP headers, forms, all you need to mimic an internet browser.
cURL main site,
cURL download page.

Unix/Linux: build cURL from the sources (compile with openssl), download a package for your distribution (or use apt-get)
Windows: Get a version that support the HTTPS protocole

In action

The Perl script is performing the following actions:

send a login SOAP call to Salesforce.com
parse the response and get the session id
send a SOAP call that will execute the Apex command

Your Salesforce.com org data model. Add to the Lead object the custom field:

Name: phone cti
API Name: phone_cti__c
type: text(50)
Read: all profiles
Write: only the System Administrator

Apex Batch

This batch is making a copy of the Lead phone field to the phone_cti__c field and apply a filter that removes all the none numeric characters.
This is just some dummy code written for this recipe and not a CTI best practice!
Here is the Apex code, test method is nested in the class:

/*
BatchLeadPhones Blp = new BatchLeadPhones();
ID batchprocessid = Database.executeBatch(Blp);
System.Debug('####batchprocessid='+batchprocessid);
*/
global class BatchLeadPhones implements Database.Batchable{
    public String query;
    global database.querylocator start(Database.BatchableContext BC){
        if ((query==null) || (query=='')) query='Select phone_cti__c From Lead';
        return Database.getQueryLocator(query);
    }

    global void execute(Database.BatchableContext BC, List scope){
        List Leads = new List();
        for(sObject s : scope){
                Lead l = (Lead)s;
                String ph = l.phone_cti__c.replaceAll('[^\\d]*','');
                if (ph != l.phone_cti__c) {
                    l.phone_cti__c = ph;
                    Leads.Add(l);
                }
        }
        if (Leads.Size()>0) update Leads; 
    }
    
    global void finish(Database.BatchableContext BC){
    }
    
    // test method
    static testMethod void test_BatchLeadPhones() {
        Lead l = new lead(LastName='lead test 123', phone_cti__c='+1 (555)123-4567', company='test company 456');
        insert l;
        Test.StartTest();
        BatchLeadPhones Blp = new BatchLeadPhones();
        Blp.query = 'Select phone_cti__c From lead Where ID=\''+l.Id+'\'';
        ID batchprocessid = Database.executeBatch(Blp);
        Test.StopTest(); 
    }
}

Warning, there is a bug in the SyntaxHighlighter module that forces tags to be closed. Do not pay attention to the last line.

Calling the Batch from the System Log window, run on all leads:

BatchLeadPhones Blp = new BatchLeadPhones();
ID batchprocessid = Database.executeBatch(Blp);
System.Debug('####batchprocessid='+batchprocessid);

Run on a single lead:

BatchLeadPhones Blp = new BatchLeadPhones();
Blp.query = 'Select phone, phone_cti__c From lead Where ID=\'00QA000000GCVof\'';
ID batchprocessid = Database.executeBatch(Blp);
System.Debug('####batchprocessid='+batchprocessid);

Calling the Batch from a web service:
This is the Apex class exposed as a webservice:

global class callBatches {
    WebService static String CallBatchLeadPhones() {
        BatchLeadPhones Blp = new BatchLeadPhones();
        ID batchprocessid = Database.executeBatch(Blp);
        return batchprocessid;
    }

    // Test Method
    TestMethod static void test_CallBatchLeadPhones() {
        Lead l = new lead(LastName='lead test 123', phone_cti__c='+1 (555)123-4567', company='test company 456');
        insert l;
        String ID = callBatches.CallBatchLeadPhones();
        System.assert(ID != null);
    }
}

Now you need the WSDL file.

Go to Setup | App Setup | Develop | Apex Classes. On the row of the class callBatches, click on the link “WSDL” to download the WSDL. This XML file describes how to call the web service.

As I guess you are not fluent in WSDL, I recommend that you install SOAPUI on your system. SOAPUI is available as a free edition for gnu/Linux, Windows and Mac OSX; binary and source code.
This recipe is not a SOAPUI tutorial; I will assume that you are familiar with this fantastic tool.
You might want to provide a partner WSDL to SOAPUI and get the SOAP message to open a session in Salesforce.com. Here is the body:

login.xml




user@domain.com
password+token

Replace the username, password and token by your own credentials.
Salesforce will return a SOAP message containing informations regarding the user settings and the organization. The session id appears here:

00DA0000000AXPZ!AQYAQC1_3X1zuSc47y75CU5a4omSypSox6Bg.j.hIsGDBv9hnc7b9ZAD.98ZST3jYxwqoY5TyF4VR7YDUxfWn.ZmeDnoY1Nv

Try this call using cURL. Open a shell:

curl --insecure --silent https://login.salesforce.com/services/Soap/u/21.0 -H "Content-Type: text/xml;charset=UTF-8" -H "SOAPAction: login" -d @login.xml > loginresponse.xml

The SOAP response is saved to loginresponse.xml.
Remark: the --insecure parameter tells cURL not to check the peer. By default, cURL is always checking. See this page to understand how to work with certificates:

Create a new SOAPUI project with your Apex class WSDL, fill the “?” parameters. The set of values is documented in the WSDL file.

The SOAP message should look like:


   
      
         true
      
      
         
         
            Apex_code
            Debug
         
         Debugonly
      
      
         
      
      
         00DA0000000AXPZ!AQYAQC1_3X1zuSc47y75CU5a4omSypSox6Bg.j.hIsGDBv9hnc7b9ZAD.98ZST3jYxwqoY5TyF4VR7YDUxfWn.ZmeDnoY1Nv

Salesforce.com will answer:


   
      
         21.0 APEX_CODE,DEBUG
23:04:07.035|EXECUTION_STARTED
23:04:07.035|CODE_UNIT_STARTED|[EXTERNAL]|01pA0000002mr28|callBatches.CallBatchLeadPhones
23:04:07.035|METHOD_ENTRY|[1]|01pA0000002mr28|callBatches.callBatches()
23:04:07.035|METHOD_EXIT|[1]|callBatches
23:04:07.042|METHOD_ENTRY|[6]|01pA0000002meNJ|BatchLeadPhones.BatchLeadPhones()
23:04:07.042|METHOD_EXIT|[6]|BatchLeadPhones
23:04:07.042|CONSTRUCTOR_ENTRY|[4]|01pA0000002meNJ|<init>()
23:04:07.042|CONSTRUCTOR_EXIT|[4]|<init>()
23:04:07.042|METHOD_ENTRY|[5]|Database.executeBatch(APEX_OBJECT)
23:04:07.085|METHOD_EXIT|[5]|Database.executeBatch(APEX_OBJECT)
23:04:07.090|CODE_UNIT_FINISHED|callBatches.CallBatchLeadPhones
23:04:07.090|EXECUTION_FINISHED
      
   
   
      
         707A000000Cj9dKIAR

4) Solution Perl Script

The Perl script is making the same actions.

login call (login.xml)
get the session id
insert the session id in the Apex method web service call
call the Apex method (request1_tpl.xml)

request1_tpl.xml is a template SOAP call. The session Id is represented by:

#ID#

Perl is replacing #ID# by the value of the session ID and saves the file to request1.xml.
Remember to update the instance name on line 16 (na1, na2, eu1, etc.)

#!/usr/bin/perl -w
use strict;

system('> loginresponse.xml');
system('curl --insecure --silent https://login.salesforce.com/services/Soap/u/21.0 -H "Content-Type: text/xml;charset=UTF-8" -H "SOAPAction: login" -d @login.xml > loginresponse.xml');
open (FILE, 'loginresponse.xml') or die ('cannot read loginresponse.xml');
my $xml=''; while() { $xml.=$_; } close FILE;
if ($xml=~m|([0-9a-z!_\.-]+)|i) {
 my $sessionid=$1;
 open (FILER, 'request1_tpl.xml') or die ('cannot read request1_tpl.xml');
 $xml=''; while() { $xml.=$_; } close FILER;
 $xml=~s/#ID#/$sessionid/;
 open (FILEW, '> request1.xml') or die ('cannot write to request1.xml');
 print FILEW $xml;
 close FILEW;
 system('curl --insecure --silent https://eu1-api.salesforce.com/services/Soap/class/callBatches -H "Content-Type: text/xml;charset=UTF-8" -H "SOAPAction: SessionHeader" -d @request1.xml > response1.xml');
}
exit;

Warning, there is a bug in the SyntaxHighlighter module that forces tags to be closed. Do not pay attention to the last line.

Remark: the command "system('> loginresponse.xml');" is not working on a Windows system. Replace it by "system('echo . 2> loginresponse.xml');"

5) Source code

source code (zip)

Monday, April 25, 2011

[Salesforce.com] The Code Review Tool Kit

Today I want to share a set of shell scripts that I am using when working on my code reviews.

Code review, the big picture
It consists in performing a validation of the work done by a partner, checking:
- technical design,
- code quality, respect of the best practices,
- governor limits,
- etc.

Context
Too often, I do not have an Internet access on the customer site. I am retrieving the Salesforce organization meta data using eclipse and my 3G key; then I am working off line.

What do the scripts?
On the first hand, the scripts provide a high level overview of the code structure: how many classes, pages, components, triggers, where are the classes used, etc. On the other hand two additional tools: find every usage in the org of a field and get all the relationships between the objects (lookup and master/detail).
All the scripts generate a CSV output for an easy integration in Excel, copy/paste from the shell to Excel! (I like this motto)
This is really save my time and let me focus on the review analysis.

Script sources
The code is available under the GPL license 3.0

	apex.sh Where my class is used? Generate a CSV text output that provides an overview of each class usage. Rows: class Cols: pages, components, triggers
	vf.sh On which classes and components does my page rely on? Generate a CSV text output that provides the list of classes, components and objects (standard controller) used by each page. Rows: page Cols: standard controller, controller, extensions
	trigger.sh Which classes are called by my trigger? Generate a CSV text output that provides the list of classes used by each trigger. Rows: trigger Cols: classes
	component.sh On which classes does my component rely on? Which pages include my component? Generate a CSV text output that provides an overview of classes used by each component, and also the pages that are using this component. Rows: component Cols: controller, extensions, pages
	field_usage.sh Where is my field used? Generate a CSV text output that provides the list of meta data where the provided field is used.
	objects_dep.sh What are the relationships between my objects? Generate a CSV text output that provides every relationship (lookup/master detail) pointing to another object.

SOLUTION DISCUSSION
The requirements
My scripts could be replaced by a program that communicates with the API and uses the DescribeObject method. But just keep in mind my two requirements:
- an off line usage
- Keep It Simple

Why a shell script?
The scripts might be optimized, or would be nicer written in a language such as Perl or Ruby rather than a shell script. I experimented several Perl XML parsers but they were slow and required Perl and Unix skills to be compiled, too complex to be shared. The choice of shell scripts appears as the best solution, with an easy deployment regardless the platform:
- Unix or gnu/Linux shell
- Mac OSX shell
- Windows using Cygwin
Awk is also required (provided with these environments).
My working place is gnu/Linux, Ubuntu flavor.

Fill the spaces!
Nature abhors a vacuum. I recommend not using blank spaces in your Eclipse project names. Always prefer an underscore. This is really making sense when it comes to shell scripts. In a shell, all space characters contained in file or directory names should be protected with a backslash:
$ cd Force.com\ IDE
Same issue in a for loop: e.g. the string “Force.com IDE” is processed as two separate files/directories:
$ for i in `echo "Force.com IDE"`; do echo $i; done
Force.com
IDE
The solution is creating symbolic links:
olivier@Ubuntu1:~/Workspaces$ ln -s Force.com\ IDE Force.com_IDE

olivier@Ubuntu1:~/Workspaces$ ls -l
total 4
drwxr-xr-x 21 olivier olivier 4096 2011-03-05 09:55 Force.com IDE
lrwxrwxrwx 1 olivier olivier 13 2011-01-26 10:43 Force.com_IDE -> Force.com IDE

Remark 1: my Eclipse workspaces path is ~/Workspaces
All my projects are located under “~/Workspaces/Force.com_IDE”

Remark 2: the provided scripts will work even if your files/directories names contains spaces because I protected the paths with double quotes.

Preparing the environment
Unzip the archive scripts.tgz and copy the scripts to your “Force.com_IDE” directory, that is where are located your projects directories. This is convenient, the scripts will be able to reach any of your eclipse projects.
To run the scripts, you have to download at least the following meta data:
- classes
- pages
- triggers
- components
- objects
Remark: all the meta data are required for the script field_usage.sh.
To add more meta data to your existing Eclipse project: open the project properties window:
click on the “Add/Remove” button:

choose your meta data:

Code naming convention
I am working with this naming convention:
Visual force pages: VF + [number] + [name].
Triggers: [object name] + [event] + [name]. The name is optional
Apex classes (trigger): AP + [number] + name
Apex classes (controller): VF + [number] + name + _Ctrl. The class name should match with the Visual force page name
Apex classes (others): name
Components: CO + [number] + [name].
Remark: the number are defined in your technical design document (assuming you have one).

Scripts “limitation”
The best approach would had been to work with a code parser, a XML parser and multi lines regular expressions. I choose a “quick & dirty” way, using simple shell commands line:
Pro's
- quicker development
- easy to understand, maintain
- very fast script execution
con's
- hell, no con's, that doing the job pretty well!

THE SCRIPTS
./apex.sh
Run the command:
apex.sh [Directory]
Result:
The script prints to the standard output a CSV text result. Each row represents a class and provides the list of the pages, components and triggers where the class is used:
When the class is used in several vf pages, the names are separated by a pipe sign.

Here is a command line sample, where Dummy_Org is the name of my Salesforce organization.
olivier@Ubuntu1:~/Workspaces/Force.com_IDE$ ./apex.sh Dummy_Org
class,page,component,trigger
AP01_OpportunityStatus,,,OpportunityAfterUpdate
AP02_AccountHierarchy,,,AccountBeforeCreateUpdate
AP03_OpportunityLost,,,OpportunityAfterUpdate
CO01_mashup_Ctrl,,CO01_mashup,
VF01_AccountNew_Ctrl,VF01_AccountNew,,
VF02_ContractWizard_Ctrl,VF02_ContractWizard,,
VF03_AccountList,VF03_AccountList,,

You might wish to copy/paste the result to Excel and get a fancy layout:

vf.sh
Run the command:
./vf.sh [Directory]
Result:
The script prints to the standard output a CSV text result. Each row represents a visual Force page and provides the list of the Object controller, Apex controller, Apex extension used by the page.

Here is a command line sample:
olivier@Ubuntu1:~/Workspaces/Force.com_IDE$ ./vf.sh Dummy_Org
page,stdcontroller,controller,extensions
VF01_AccountNew,,VF01_AccountNew_Ctrl,
VF02_ContractWizard,,VF02_ContractWizard_Ctrl,
VF03_AccountList,Account,,VF03_AccountList

trigger.sh
Run the command:
./trigger.sh [Directory]
Result:
The script prints to the standard output a CSV text result. Each row represents a trigger and provides the list of the Apex classes used by the trigger:

Here is a command line sample:
olivier@Ubuntu1:~/Workspaces/Force.com_IDE$ ./trigger.sh Dummy_Org
trigger,classes
AccountBeforeCreateUpdate,AP02_AccountHierarchy
OpportunityAfterUpdate,AP01_OpportunityStatus|AP03_OpportunityLost

In this example, the trigger OpportunityAfterUpdate is calling two classes: AP01_OpportunityStatus and AP03_OpportunityLost

components.sh
Run the command:
./components.sh [Directory]
Result:
The script prints to the standard output a CSV text result. Each row represents a component and provides the list of the Apex controller, Apex extension used by the component and the visual force pages where the component is used.

Here is a command line sample:
olivier@Ubuntu1:~/Workspaces/Force.com_IDE$ ./components.sh Dummy_Org
component,controller,extension,page
CO01_mashup,CO01_mashup_Ctrl,,

field_usage.sh
Run the command:
./field_usage.sh [Directory] [field name]
Result:
The script prints to the standard output a CSV text result. Each row represents a meta object where the search field is present.
Columns: object type, object name, number of matches.

Here is a command line sample:
olivier@Ubuntu1:~/Workspaces/Force.com_IDE$ ./field_usage.sh Dummy_Org Segmentation__c
metaObject,name,occurence
applications,Dummy_Org_App,1
classes,VF01_AccountNew_Ctrl,10
classes,VF02_ContractWizard_Ctrl,8
layouts,Account-Customer Layout,1
layouts,Account-Prospect Layout,1
objects,Account,3
objects,Site__c,7
profiles,Account Manager,11
profiles,Admin,12
profiles,ContractManager,11
profiles,MarketingProfile,11
profiles,ReadOnly,11
profiles,Standard,11
triggers,AccountBeforeCreateUpdate,1

Remark 1: as the search relies only on the name, the script will return every occurrence of the file name. E.g. if you have created one custom field “segmentation__c” on the account object and another one on the opportunity object, all the occurrences will be returned, regardless their parent object.

Remark 2: I do not remove the comments in the Apex and Visual force (this will come in a future release). So when the field name appears in a comment it is considered as a valuable match.

objects_dep.sh
Run the command:
./objects_dep.sh [Directory]
Result:
The script prints to the standard output a CSV text result. Each row represents a relationship.
Columns: object 1, type of relationship, object 2.
object 1 as a relationship pointing to object 2.
the type of relationship might be “lookup” for a lookup, and “md” for a master-detail.

Here is a command line sample:
olivier@Ubuntu1:~/Workspaces/Force.com_IDE$ ./objects_dep.sh Dummy_Org
parent,relationship,child
Account,lookup,AccountExecutive__c
Account,lookup,CountryId__c
Contact,lookup,CountryId__c
Town__c,md,CountryId__c

Conclusion
I hope you will find these scripts useful. I am using them almost every day, so I will bring improvements and will post updated versions to this blog. Feel free to give your feedback by dropping a note in the blog comments.