Team Blog http://www.simego.com/Blog Blog articles from the Simego team relating to Simego products and other software technology http://www.simego.com/Blog/2014/07/Dynamics-CRM-Audit-History-Download david@simego.com (David Evans) Dynamics CRM Audit History Download <p>Dynamics CRM 2013 and Dynamics CRM Online both have excellent auditing capabilities. Unfortunately, neither platform allows the extract of the full audit trail in bulk but require explicit item by item extraction. There are several methods of extracting the details directly out of the on-premise Dynamics CRM SQL Server but I would always shy away from these solutions as they are unsupported by Microsoft.</p> <p>&nbsp;</p> <p>The solution I am going to present requires 3 stages</p> <p>&nbsp;</p> <ol> <li><span style="line-height: 25px;">Extraction of the core &lsquo;Audit&rsquo; entity into a SQL Server table.</span></li> <li><span style="line-height: 25px;">For each Audit entity the extraction of the detailed audit record into a &lsquo;Dynamic Column&rsquo;.</span></li> <li><span style="line-height: 25px;">Filtering out subsequent requests so that we do not repeatedly extract the same audit detail unnecessarily.</span></li> </ol> <p>&nbsp;</p> <p><strong>Step 1: Extraction of the core &lsquo;Audit&rsquo; entity into a SQL Server table.</strong></p> <p>&nbsp;</p> <p>Open the Connect To Data Source dialog and complete the details for you Dynamics CRM Online instance.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_094c4a1f-d747-42f7-8306-715a0b2c708b"><img height="570" width="1028" src="https://simegoimages.blob.core.windows.net/images/blog_547a3cf9-542b-447e-9b20-62043d5b6383" alt="Connecting to Dynamics CRM Online" style="display: inline; border: 0px;" title="Connecting to Dynamics CRM Online" /></a></p> <p>&nbsp;</p> <p>Once you have connected DS3 will display all of the available attributes in the Data Source (A) tree.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_a36a02f3-da55-4a8f-acfe-40ba712e6f10"><img height="520" width="733" src="https://simegoimages.blob.core.windows.net/images/blog_2564102a-7e8b-46d6-9216-24192d544a64" alt="Extracting the Dynamics CRM Audit History entity schema" style="display: inline; border: 0px;" title="Extracting the Dynamics CRM Audit History entity schema" /></a></p> <p>&nbsp;</p> <p>We are going to add a new Dynamic Column called auditdetails to the source giving our complete source schema.</p> <p>&nbsp;</p> <p><strong>Step 2: For each Audit entity the extraction of the detailed audit record into a &lsquo;Dynamic Column&rsquo;.</strong></p> <p>&nbsp;</p> <p>To add a Dynamic Column you first need to enable the Dynamic Column feature. From the View menu select the &lsquo;Dynamic Columns Window&rsquo;. This will present you with the dynamic Columns tab. On this table click on the &lsquo;Enable Dynamic Columns&rsquo; Link.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_82257cb1-be66-48ec-9d28-374d572f91bd"><img height="520" width="733" src="https://simegoimages.blob.core.windows.net/images/blog_eda63975-d500-40cf-bdce-f16afa8e1aa7" alt="DS3 Dynamics Columns" style="display: inline; border: 0px;" title="DS3 Dynamics Columns" /></a></p> <p>&nbsp;</p> <p>Dynamic Columns are a very special feature which allows users to create new columns from code. The code can re-use the source and target connections as well as the values of the current row being connected to.&nbsp; When you enable the feature you get a default code template.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_61b132db-bcb4-4032-a770-dae12eb92aac"><img height="520" width="733" src="https://simegoimages.blob.core.windows.net/images/blog_b8df7603-428b-434c-a697-27f055a0756e" alt="DS3 Dynamics Columns template" style="display: inline; border: 0px;" title="DS3 Dynamics Columns template" /></a></p> <p>&nbsp;</p> <p>If you are familiar with c# code this will be no surprise. We are going to use some SDK code to extract the audit details for each row but only if their has been a change.</p> <p>&nbsp;</p> <p>Firstly we are going to add some references to the necessary Dynamic CRM SDK&rsquo;s:</p> <p>&nbsp;</p> <pre class="csharpcode"><span class="kwrd">using</span> Microsoft.Xrm.Sdk; <span class="kwrd">using</span> Microsoft.Xrm.Sdk.Query; <span class="kwrd">using</span> Microsoft.Xrm.Sdk.Client; <span class="kwrd">using</span> Microsoft.Xrm.Sdk.Messages; <span class="kwrd">using</span> Microsoft.Xrm.Sdk.Metadata; <span class="kwrd">using</span> System.Runtime.Serialization; <span class="kwrd">using</span> System.Xml; <span class="kwrd">using</span> System.IO; <span class="rem">// These namespaces are found in the Microsoft.Crm.Sdk.Proxy.dll assembly</span> <span class="rem">// located in the SDK\bin folder of the SDK download.</span> <span class="kwrd">using</span> Microsoft.Crm.Sdk.Messages;</pre> <style type="text/css"><!-- .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } --></style> <p>&nbsp;</p> <p>In between the BeginRow() and Setup() methods we are going to add the following code.</p> <p>&nbsp;</p> <p>The code essentially exposes the Audit change details as a property called auditdetails which returns into an xml formatted string which we can store in the database.</p> <p>&nbsp;</p> <pre class="csharpcode"><span class="kwrd">public</span> <span class="kwrd">string</span> auditdetails { get { <span class="rem">//Only get the audit details if there is a create, update or delete</span> <span class="kwrd">if</span>(actionname ==<span class="str">"Update"</span> | actionname ==<span class="str">"Create"</span>| actionname ==<span class="str">"Delete"</span>) { Trace.WriteLine(<span class="str">"Row {0}"</span>,NEXTINT()); <span class="rem">//Use the current connection</span> Simego.DataSync.Providers.MSCrm2011.MSCrmDataSourceReader reader = (Simego.DataSync.Providers.MSCrm2011.MSCrmDataSourceReader) <span class="kwrd">this</span>.DataSourceA.Reader; OrganizationServiceProxy orgService = reader.Connection.GetProxy(); <span class="rem">//Create a history request</span> RetrieveRecordChangeHistoryRequest changeRequest = <span class="kwrd">new</span> RetrieveRecordChangeHistoryRequest(); changeRequest.Target = <span class="kwrd">new</span> EntityReference(<span class="kwrd">this</span>.objecttypecode, <span class="kwrd">this</span>.objectid); <span class="rem">//Get the history request from the CRM system</span> RetrieveRecordChangeHistoryResponse changeResponse = (RetrieveRecordChangeHistoryResponse)orgService.Execute(changeRequest); <span class="rem">//Serialise the response into an xml string</span> <span class="kwrd">using</span> (var stringWriter = <span class="kwrd">new</span> StringWriter()) { XmlWriterSettings settings = <span class="kwrd">new</span> XmlWriterSettings { Indent = <span class="kwrd">true</span> }; <span class="kwrd">using</span> (var writer = XmlWriter.Create(stringWriter,settings)) { var contractSerializer = <span class="kwrd">new</span> DataContractSerializer(<span class="kwrd">typeof</span>(RetrieveRecordChangeHistoryResponse)); contractSerializer.WriteObject(writer,changeResponse); } <span class="kwrd">return</span> stringWriter.ToString(); } } <span class="rem">//return nothing if it is not create, update or delete</span> <span class="kwrd">return</span><span class="str">""</span>; } }</pre> <style type="text/css"><!-- .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } --></style> <p>&nbsp;</p> <p>This result is essentially the exact contents of a &lsquo;RetrieveRecordChangeHistoryResponse&lsquo; as defined by the Dynamics CRM SDK. Why did we do this? We can then at a later date extract the full details of the change by loading it back into a RetrieveRecordChangeHistoryResponse (using serialize and deserialize).</p> <p>&nbsp;</p> <p>Pressing the Build button converts the auditdetails code into a column</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d2659933-a92d-4d9c-a498-a6da698a93a3"><img height="717" width="924" src="https://simegoimages.blob.core.windows.net/images/blog_a4bc87f6-dd30-418e-8d31-86b65d282a34" alt="Dynamics CRM Audit Details extracted into a column" style="display: inline; border: 0px;" title="Dynamics CRM Audit Details extracted into a column" /></a></p> <p>&nbsp;</p> <p>We can now use another of DS3&rsquo;s unique features which will create a SQL table exactly the same schema as the Data Source (A). From the tools menu select Tools &ndash;&gt; Create SQL Table. You will be prompted through a wizard, which will eventually create a SQL Table in a SQL Server and connect Data Source (B) to it.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_3b218921-eb16-4143-85ee-944aca4cdb8f"><img height="383" width="519" src="https://simegoimages.blob.core.windows.net/images/blog_b2e01419-d5bb-4f44-82a2-1364e610596b" alt="Create a SQL Table from Dynamics CRM Schema" style="display: inline; border: 0px;" title="Create a SQL Table from Dynamics CRM Schema" /></a></p> <p>&nbsp;</p> <p>Once connected the SQL Server Table, the schemas should automatically map themselves together.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_acaa6a5a-ff3b-4ec9-8bea-74bd2b533655"><img height="838" width="1240" src="https://simegoimages.blob.core.windows.net/images/blog_b673de96-2114-4e41-a6e8-407c8a858aeb" alt="Dynamics CRM Integration with SQL Server Table for Audit History" style="display: inline; border: 0px;" title="Dynamics CRM Integration with SQL Server Table for Audit History" /></a></p> <p>&nbsp;</p> <p>We could now read all of the details from the audit entity and write them to the SQL Table. If we consider the probable size of the audit table this may be an extremely long process to repeat unnecessarily. So we need to implement some way of telling DS3 that we have already written the audit record to the local SQL Server so there is no point in calling back to Dynamics CRM again. We can do this by using a lookup into the target SQL Audit Database.</p> <p>&nbsp;</p> <p><strong>Step 3: Filtering out subsequent requests so that we do not repeatedly extract the same audit detail unnecessarily.</strong></p> <p>&nbsp;</p> <p>&nbsp;</p> <p>&nbsp;</p> <p>For each row we need to lookup into the Data Source(B) whether we have already extracted the audit record. We can achieve this by creating another &lsquo;Calculated Column&rsquo; called &lsquo;AuditValueInDatabase&rsquo; and using the LOOKUPB function. Essentially, this tells DS3 to find the auditid value from the CRMAudit table where the audit id in CRM matches the Auditid in the database. If the result is has no value then we can skip extracting the record.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d5c0d040-22e1-4b0e-a539-013bece3f82a"><img height="330" width="613" src="https://simegoimages.blob.core.windows.net/images/blog_e477c432-10c6-49a6-bb5a-f8690d5b7f5c" alt="Calculated column for looking up values in SQL Table" style="display: inline; border: 0px;" title="Calculated column for looking up values in SQL Table" /></a></p> <p>&nbsp;</p> <p>The next step is to now filter the results to only include items we cannot find in the target SQL Table. This is easy as DS3 has a filter feature which we can set to when &lsquo;AuditValueInDatabase&rsquo; is null (nothing returned).</p> <p>&nbsp;</p> <pre class="csharpcode">ISNULL(AuditValueInDatabase)</pre> <style type="text/css"><!-- .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } --></style> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_194dd2b5-7735-4e37-a3b0-613b2ce66b61"><img height="246" width="433" src="https://simegoimages.blob.core.windows.net/images/blog_df1e1c96-b556-4ae0-ac80-7f706f1a39c8" alt="Filtering out the previously recorded audit items" style="display: inline; border: 0px;" title="Filtering out the previously recorded audit items" /></a></p> <p>&nbsp;</p> <p>We are almost finished our solution. Ideally we need to keep extracting the audit trail to keep it up to date rather than extracting the whole audit table every day. We will use the &lsquo;FetchXMLFilterExpression&rsquo;&nbsp; feature to extract only the last 2 days audit entries. We can run this extract every day and in the case that we do not run it for a day it will catch up without writing duplicates as DS3 will reconcile the changes first before writing any records. The fetchxml looks like:</p> <p>&nbsp;</p> <pre class="csharpcode"><span class="kwrd">&lt;</span><span class="html">filter</span> <span class="attr">type</span><span class="kwrd">="and"</span><span class="kwrd">&gt;</span> <span class="kwrd">&lt;</span><span class="html">condition</span> <span class="attr">attribute</span><span class="kwrd">="createdon"</span> <span class="attr">operator</span><span class="kwrd">="last-x-days"</span> <span class="attr">value</span><span class="kwrd">="2"</span> <span class="kwrd">/&gt;</span> <span class="kwrd">&lt;/</span><span class="html">filter</span><span class="kwrd">&gt;</span> </pre> <style type="text/css"><!-- .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } --></style> <p>&nbsp;</p> <p>We can now run our complete Dynamics CRM Audit download project every day.</p> Team Blog Wed, 16 Jul 2014 11:17:51 +0000 http://www.simego.com/Blog/2014/07/Business-Intelligence-Dynamics-CRM-Getting-the-Data david@simego.com (David Evans) Business Intelligence + Dynamics CRM, getting the Data <p>Being able to view dashboards and mine you Dynamics CRM data is of utmost importance to most organisations. Dynamics CRM has great features allowing you to visualise your data using Dashboards, charts and other graphics.</p> <p>&nbsp;</p> <p>Sometimes though Dynamics CRM data needs to join up with data from other systems. This often requires you to extract that latest data from Dynamics CRM and load it closer to your data warehouse.</p> <p>&nbsp;</p> <p>This blog quickly demonstrates using DS3 to extract a set of entities from Dynamics CRM and schedule the continuous download of changes.</p> <p>&nbsp;</p> <p>Open DS3 Automation Server and create an empty project.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_567b487a-49c9-4a98-9fb2-ce32a0ed43bb" title="Create a new Dynamics CRM Integration project"><img height="615" width="902" src="https://simegoimages.blob.core.windows.net/images/blog_e3ce7b7d-1059-4c34-be7f-fe9404631247" alt="Creating a new Integration project" border="0" style="display: inline; border-width: 0px;" title="Creating a new Integration project" /></a></p> <p>This empty project will hold the steps that will extract the data from each entity in CRM.</p> <p>&nbsp;</p> <p>Open the Project and add a Data Synchronisation Step.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_0cf034de-cbe1-4115-a6c6-295f1a1115fb" title="Select an integration Step type"><img height="912" width="1338" src="https://simegoimages.blob.core.windows.net/images/blog_af54c6e7-1d7c-4c20-93ae-de3f324f8c3f" alt="Choosing a new Simego Project Step" border="0" style="display: inline; border-width: 0px;" title="Choosing a new Simego Project Step" /></a></p> <p>&nbsp;</p> <p>Create and document the step and then click Save. An empty integration project will be created so you can open and get editing easily.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_cd6f2da3-151e-4030-945d-0c44ee37386f" title="Edit the Integration Step"><img height="671" width="922" src="https://simegoimages.blob.core.windows.net/images/blog_074e7c41-78ee-48f1-8d4c-b6122f6d6be4" alt="Editing the new Step" border="0" style="display: inline; border-width: 0px;" title="Editing the new Step" /></a></p> <p>Once you have saved the step, open the integration definition by clicking &lsquo;Open File&rsquo;.</p> <p>&nbsp;</p> <p>This will launch DS3 Developer when we can define the integration source Dynamic CRM entity, SQL Table Target and mapping.</p> <p>&nbsp;</p> <p>Navigate through you connections tree and drag and drop the account entity onto the Data Source (alternatively you can right-click and select &lsquo;Connect to DataSource (A)&rsquo;</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_cd03adeb-d004-44f3-9daa-434f095d7469" title="Create the Integration mapping from the Dynamics CRM Entity with no Code"><img height="486" width="523" src="https://simegoimages.blob.core.windows.net/images/blog_e95c3afc-60bd-47d2-8cf1-d7255fdd7459" alt="Setting the Dynamics CRM Integration source" border="0" style="display: inline; border-width: 0px;" title="Setting the Dynamics CRM Integration source" /></a></p> <p>&nbsp;</p> <p>All the Account entity fields in Dynamics CRM will now be available for integration.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_a6450bc2-8190-469d-8062-3fc450c514cd" title="Preview the CRM Entities"><img height="619" width="526" src="https://simegoimages.blob.core.windows.net/images/blog_bb296f53-f5c1-4845-9c82-68b59b99f4b0" alt="Selecting the Dynamics CRM Entity attributes" border="0" style="display: inline; border-width: 0px;" title="Selecting the Dynamics CRM Entity attributes" /></a></p> <p>&nbsp;</p> <p>Tick beside each field you want extracted from the entity, this will add them to the source schema.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_1dbeed9e-dc34-4231-9083-f35ffdb3f9e1" title="Select the Dynamics CRM Attributes"><img height="539" width="746" src="https://simegoimages.blob.core.windows.net/images/blog_0d238e07-d4b7-4370-be1b-1f0edc14b86d" alt="Setying the primary ket for the integration" border="0" style="display: inline; border-width: 0px;" title="Setying the primary ket for the integration" /></a></p> <p>&nbsp;</p> <p>Using the tools menu you can as DS3 to create a SQL Table from the exact list of fields you have chosen.</p> <p>&nbsp;</p> <p>Open the Tools menu and click the&nbsp; &lsquo;Create SQL Table&rsquo; option.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_b58f4770-0fab-492e-a2d1-b541ef084bb7" title="Business Intelligence + Dynamics CRM relies on having a target data source, in this case a SQL Table"><img height="854" width="1182" src="https://simegoimages.blob.core.windows.net/images/blog_71e00dd2-181a-4409-9c2c-530a837b5afd" alt="Automatically creating the SQL Table to import the Dynamics CRM Entity data into" border="0" style="display: inline; border-width: 0px;" title="Automatically creating the SQL Table to import the Dynamics CRM Entity data into" /></a></p> <p>&nbsp;</p> <p>The SQL Table Wizard will appear. This will guide you though creating a SQL Table with the same schema as the source CRM Entity.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_b3e4ae47-a0c9-4624-9e59-346ace38126f" title="Create SQL Table Wizard "><img height="377" width="515" src="https://simegoimages.blob.core.windows.net/images/blog_4a7a14ac-2684-4d31-95fa-bf84b8742138" alt="Simego SQL Table Wizard" border="0" style="display: inline; border-width: 0px;" title="Simego SQL Table Wizard" /></a></p> <p>&nbsp;</p> <p>At the end of the wizard, DS3 will use the newly created table as the target.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ac55b645-e055-4a30-bf2a-42330e79be2b" title="Complete mapping from Dynamics CRM Entity to the SQL Table for Business Intelligence"><img height="760" width="1320" src="https://simegoimages.blob.core.windows.net/images/blog_3ff97760-011d-4af2-a867-ce55093f79aa" alt="Dynamics CRM Integration with SQL mapping" border="0" style="display: inline; border-width: 0px;" title="Dynamics CRM Integration with SQL mapping" /></a></p> <p>&nbsp;</p> <p>To verify if the integration is working, click on the Compare A-&gt;B and you can then preview the records that will come across.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_e1c05848-7039-4cd8-846b-96429dd5a95b" title="Preview the Data being integrated "><img height="470" width="960" src="https://simegoimages.blob.core.windows.net/images/blog_82242b82-32fc-4c68-8eaf-0d048534afd1" alt="Prevewi of integration items" border="0" style="display: inline; border-width: 0px;" title="Prevewi of integration items" /></a></p> <p>&nbsp;</p> <p>Now you can just save the project back to the Automation server by clicking &lsquo;Save&rsquo;.</p> <p>&nbsp;</p> <p>Back in the automation server you can just run the project by click &lsquo;Start Project&rsquo;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_c1e3ccaf-dc4f-49ff-9471-351507c9a566" title="Automated the CRM Extract"><img height="493" width="1471" src="https://simegoimages.blob.core.windows.net/images/blog_be1d60fb-14e2-4942-952c-3a643fee3a3b" alt="Automating the CRM Data Extract" border="0" style="display: inline; border-width: 0px;" title="Automating the CRM Data Extract" /></a></p> <p>&nbsp;</p> <p>You can view the progress on the &lsquo;Reporting&rsquo; page in real-time.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_4b59828b-40f0-4200-957b-1b304077031c" title="Monitor the Extract"><img height="433" width="669" src="https://simegoimages.blob.core.windows.net/images/blog_f7d1b409-ff21-442b-8f29-26438b887d53" alt="Running the Dynamics CRM Integration" border="0" style="display: inline; border-width: 0px;" title="Running the Dynamics CRM Integration" /></a></p> <p>&nbsp;</p> <p>&hellip;and when it completes successfully.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d76cb747-15ce-4397-aaac-9b6c36f772af" title="Extract Completed"><img height="433" width="669" src="https://simegoimages.blob.core.windows.net/images/blog_f88f4412-72d0-4e1e-ad0e-80453133d9b1" alt="CRM Integration completed successfully" border="0" style="display: inline; border-width: 0px;" title="CRM Integration completed successfully" /></a></p> <p>&nbsp;</p> <p>Drilling down into the logs will show us some detail on the integration such as 9916 records were missing from the SQL Server and it took 25.225 seconds to insert them.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_aaa7f2bd-52ce-4a84-92ef-8734eeaaec2a" title="View Integration Log"><img height="737" width="1353" src="https://simegoimages.blob.core.windows.net/images/blog_86d92830-28da-4946-be69-e1c5a72b573b" alt="Detailed logging" border="0" style="display: inline; border-width: 0px;" title="Detailed logging" /></a></p> <p>&nbsp;</p> <p>We keep repeating the process of connecting to the Dynamic CRM entities and creating target tables until we have a set of entities to bring into the Data Warehouse.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_28072669-9f4c-45d8-b0ba-e8d60bddf03f" title="Sample CRM Extract for Business Intelligence"><img height="587" width="1345" src="https://simegoimages.blob.core.windows.net/images/blog_ddd59feb-df75-405c-b69c-724cb7ddb49f" alt="Overall Business Intelligence Extract from CRM" border="0" style="display: inline; border-width: 0px;" title="Overall Business Intelligence Extract from CRM" /></a></p> <p>&nbsp;</p> <p>We now have to set a schedule to run these extracts. The Automation server &lsquo;Triggers&rsquo; projects based on an event. We can set these events or triggers to fire based on many different criteria. We are going to use the simplest &lsquo;Time of Day&rsquo;.</p> <p>&nbsp;</p> <p>Create a new Time Trigger from the &lsquo;Triggers&rsquo; menu.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_a7782dd3-6cee-456f-998f-4fad6ea66031" title="Create the CRM Extract Schedule"><img height="626" width="914" src="https://simegoimages.blob.core.windows.net/images/blog_d85b8b7d-0535-4c26-9153-9a351187aae6" alt="Create a Schedule for the Business Intelligence extract" border="0" style="display: inline; border-width: 0px;" title="Create a Schedule for the Business Intelligence extract" /></a></p> <p>&nbsp;</p> <p>Set the time to 11 pm every week night and call it &lsquo;Late Night Weekday Trigger&rsquo;.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_14d8ad36-6cf9-481f-b020-b748d357a8ef" title="Trigger the project to run"><img height="630" width="918" src="https://simegoimages.blob.core.windows.net/images/blog_078f9288-b092-4622-aaef-8c4eb64e1011" alt="Schedule configuration" border="0" style="display: inline; border-width: 0px;" title="Schedule configuration" /></a></p> <p>&nbsp;</p> <p>Navigate to the &lsquo;Dynamics CRM Data Extract for Reporting&rsquo; project click Add Trigger from the sub menu.&nbsp; Select the late night trigger.</p> <p>&nbsp;</p> <p>Whenever we need to download an additional entity, we can just add a new integration to our steps and it will be included in the synchronisation.</p> Team Blog Wed, 02 Jul 2014 16:08:16 +0000 http://www.simego.com/Blog/2014/06/If-your-product-or-service-was-a-person--would-you-like-them david@simego.com (David Evans) If your product or service was a person, would you like them? <p>Think of the service you provide and give it a personality and a name like &lsquo;Jim&rsquo;.</p> <p>&nbsp;</p> <p>Think of yourself as a consumer of you service and product:</p> <ol> <li>Appreciate that every time you interact with &lsquo;Jim&rsquo; you form an opinion about him. </li> <li>If you &lsquo;love&rsquo; Jim, you will tell all your friends and invest your credibility in promoting Jim&rsquo;s abilities. </li> <li>If you &lsquo;hate&rsquo; Jim, you will tell all your friends and invest time in avoiding Jim or bad mouthing him. </li> <li>If you don&rsquo;t trust Jim, you will never invest time getting to know him.</li> </ol> <p>&nbsp;</p> <p>Jim can influence you by:</p> <ol> <li>Responding to questions and requests clearly and accurately. </li> <li>Not Ignoring you. </li> <li>Not Ignoring your opinion. </li> <li>Being attentive during conversations. </li> <li>Asking your opinion. </li> <li>Valuing your opinion. </li> <li>Valuing you personal information. </li> <li>Valuing your relationship. </li> <li>Being endearing. </li> <li>Apologizing when he makes a mistake. </li> <li>Introducing you to his friends that he knows you&rsquo;ll like.</li> </ol> <p>Not an exhaustive list bit you get the idea. It is important to notice that if Jim pretends to be someone his is not it will only backfire. Asking for your address so he can introduce you to his friends nearby and then telling everyone where your live is not endearing.</p> Team Blog Mon, 23 Jun 2014 13:00:34 +0000 http://www.simego.com/Blog/2014/04/Reading-Data-from-Dynamics-NAV--Navision--using-the-SOAP-Web-Service-Connector david@simego.com (David Evans) Reading Data from Dynamics NAV (Navision) using the SOAP Web Service Connector <p>SOAP Web services remain a popular method of integration even with the rise of the API and json based integration protocols. Key Microsoft products like SharePoint, Dynamics CRM and others like Navision utilise it as a clean and approachable integration point.</p> <p>&nbsp;</p> <p><strong>Navision (NAV) SOAP Web Services</strong></p> <p><strong>&nbsp;</strong>&nbsp;</p> <p>Data can be extracted from Dynamics NAV via direct SQL queries and NAV hosted web services or a combination of the both. In this post I wanted to highlight that you can extract data from Dynamics NAV using the DS3 SOAP provider and the steps needed to achieve this.</p> <p>&nbsp;</p> <p>In this example we are going to expose the Navision Customer Card (List) as a Web Service in NAV 2009 R2.</p> <p>&nbsp;</p> <p><strong>Expose / Publish the Web Service</strong></p> <p>&nbsp;</p> <p>From within NAV navigate to Administration &ndash;&gt; IT Administration &ndash;&gt; General. Then Click on the Web Services List item link</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_90506062-36bd-4777-b918-d92ee4171e75"><img title="image" style="display: inline; border: 0px;" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_b3474944-086e-4f13-9c3d-868a19de41a9" width="438" height="402" /></a></p> <p>&nbsp;</p> <p>You should see a potentially empty list of Web Services or a list of active Web Services:</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_672f76d2-3839-4def-b1d8-eb79db4001cd"><img title="image" style="display: inline; border: 0px;" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_adaa496a-b6fa-495a-bf14-6ea593c67dfa" width="670" height="590" /></a></p> <p>&nbsp;</p> <p>Click on the New button, Set the &lsquo;Object Type&rsquo; to &lsquo;Page&rsquo; and select the Object ID to 21 and name the Service Customer and tick the Publish tick box.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_c679c970-f744-4d7b-a929-bd5003a539fd"><img title="image" style="display: inline; border: 0px;" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_a9a229c8-b437-478b-9478-8ca5adffcef6" width="859" height="634" /></a></p> <p>&nbsp;</p> <p><strong>Creating the Correct Webservice URL</strong></p> <p>&nbsp;</p> <p>One of the issues with this process is when you try and find the web service endpoint url. Typically the format is:</p> <p>&nbsp;</p> <p><a href="http://servername:7047/DynamicsNAV/WS/CompanyName/services">http://<em>servername</em>:7047/DynamicsNAV/WS/<em>CompanyName</em>/services</a></p> <p>&nbsp;</p> <p>In my case for testing (<em>if you are not using Simego DS3 then you will have to replace the spaces with %20</em>):</p> <p>&nbsp;</p> <pre class="csharpcode">http://192.168.1.115:7047/DynamicsNAV/WS/CRONUS International Ltd/services</pre> <style type="text/css"><!-- .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } --></style> <p>&nbsp;</p> <p>You can open this URL and it will display the list of endpoints (Services) available from this company:</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_2d8856d6-c0b7-4ee4-b334-e696f9d28700"><img title="image" style="display: inline; border: 0px;" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_58146520-0bfb-4037-8ac2-84d54e8d9efd" width="1287" height="280" /></a></p> <p>&nbsp;</p> <p>The most important URL is the endpoint (url) pointing to the Customer service:</p> <p>&nbsp;</p> <pre class="csharpcode">http://192.168.1.115:7047/DynamicsNAV/WS/CRONUS International Ltd/Page/Customer</pre> <p>&nbsp;</p> <p>This is the endpoint we are going to use to read customer data from Navision.</p> <p>&nbsp;</p> <p><strong>Connecting to the Web Services using DS3</strong></p> <p>&nbsp;</p> <p><em>(Skip down to the animated Gif if you want to see how this is done)</em></p> <p>&nbsp;</p> <p>1. Start DS3, click the Data Source A green button and select the SOAP Services Connector.</p> <p>&nbsp;</p> <p>2. in the WebServiceURL paste in the Customer Service URL:</p> <p>&nbsp;</p> <pre class="csharpcode">http://192.168.1.115:7047/DynamicsNAV/WS/CRONUS International Ltd/Page/Customer</pre> <p> <style type="text/css"><!-- .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } --></style> </p> <p>2. Enter in the Namespace (this is not necessary but I like clean and Tidy).</p> <p>&nbsp;</p> <p>3. If you need a username and password to get to the URL then enter it in the credentials settings.</p> <p>&nbsp;</p> <p>4. Click ok and you will be presented with the Select Webservice Method dialog. We want to get a list of Customers so we navigate and select the ReadMultiple which returns an array of Customer objects. We do not need to fill in the parameters.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d6f767aa-b233-4f6f-8eaf-86df0a3550f5"><img title="image" style="display: inline; border: 0px;" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_b81c5bbf-3462-470c-9ed4-b913478cbee0" width="422" height="512" /></a></p> <p>&nbsp;</p> <p>5. Click OK and DS3 will create all the necessary &lsquo;plumbing&rsquo; to read data from the webservice.</p> <p>&nbsp;</p> <p>6. Click on the Preview A button to check whether we can retrieve the data from Navison&rsquo;s Webservice.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_78d12f8d-3c4d-4bc8-b675-32d74e4e545f"><img title="image" style="display: inline; border: 0px;" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_8ae29224-d339-43ca-90a3-0cab1ec60d9f" width="1409" height="776" /></a></p> <p>&nbsp;</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_97de9ffb-befc-4602-989e-2218e11cdeaa"><img title="image" style="display: inline; border: 0px;" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_ec383afe-42b3-489f-8de2-cfe16946122d" width="977" height="540" /></a></p> <p>&nbsp;</p> <p>7. You can now choose to connect it to another systems or just set a csv destination source using the Tools Create csv file &ndash;&gt; Compare A-&gt;B and synchronise.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ca62ad0f-2f1f-491c-a5de-49bd6b268557"><img title="Reading Data From Navision Web Servces" style="display: inline;" alt="Reading Data From Navision Web Servces" src="https://simegoimages.blob.core.windows.net/images/blog_6a851eed-4177-4c7c-8ea7-95cb684b74b1" width="1024" height="768" /></a></p> <p>&nbsp;</p> <p>Contact us if you have any questions as we are happy to demonstrate the power of DS3.</p> Team Blog Wed, 30 Apr 2014 16:07:55 +0000 http://www.simego.com/Blog/2014/04/Uploading-and-Downloading-files-using-FTP-in-Simego-DS3-Server--Ouvvi david@simego.com (David Evans) Uploading and Downloading files using FTP in Simego DS3 Server (Ouvvi) <p>FTP is a mainstay of B2B integration and whilst there may be other options to transfer files online, its ability to offer direct B2B connectivity make it a popular choice.</p> <p>&nbsp;</p> <p>FTP Upload and Download are now included in the increasing number of DS3 Server Project&nbsp; ‘Steps’.</p> <p>&nbsp;</p> <p>This post will demonstrate how to create a project which downloads an xml file from and uploads a csv file back to a FTP server. </p> <p>&nbsp;</p> <p>The FTP server is configured only to allow secure connections as this is a typical configuration option.</p> <p>&nbsp;</p> <p><strong>Create a Project to contain the FTP Steps</strong></p> <p>From the Projects Page of DS3 Server, create a new project and name it ‘FTP Integration test’.</p> <p>&nbsp;</p> <p><strong>Add a new ‘FTP Download’ Step</strong></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f960c1b5-4b8b-4a5c-b83c-9d88472255e1"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_b2b5f0ca-fa1e-436b-8804-43ae5682fdef" width="1022" height="784"></a> </p> <p>&nbsp;</p> <p>Name the step ‘Download File via FTP (s)’ and then configure it.</p> <p>&nbsp;</p> <p><strong>FTP Server File Path</strong>&nbsp; <p>This is the url style path to the file on the FTP server eg <a title="ftp://localhost/data.xml" href="ftp://localhost/data.xml"><em>ftp://localhost/data.xml</em></a> <p>&nbsp; <p><strong>FTP Server uses Passive Mode</strong> <p>This allows the FTP server to change the port the ftp process is working through. <em>This is normally enabled .</em> <p><em></em>&nbsp; <p><strong>FTP Server uses SSL</strong> <p>If the server is configured to transfer files and authentication details over SSL then you will need to enable this setting.<em>This is normally enabled .</em> <p><em></em>&nbsp; <p><strong>FTP Server accept SSL certificate</strong> <p>It is possible (and likely) that the certificate used to secure the FTP server is not signed by a trusted root authority or is not trusted by your own machine. In this case you can indicate that it is OK to accept this SSL certificate. <em>This is optionally enabled .</em> <p><strong></strong>&nbsp; <p><strong>Destination Filename</strong>&nbsp; <p>Define the name and location of the file once it has been downloaded eg <em>c:\temp\medals.xml</em> <p>&nbsp; <p><strong>Username</strong>&nbsp; <p>Set the username used to authenticate with the FTP server. <p>&nbsp; <p><strong>Password</strong> <p>Set the corresponding password to used with the Username <p>&nbsp; <p><strong>Timeout</strong> <p>This is the time in seconds that we will wait until giving up for a response from the FTP server. eg <em>Normally 30 seconds&nbsp; 00:00:30 is reasonable.</em></p> <p>&nbsp;</p> <p>Once you have completed the configuration it should something look like this. </p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_4d86b46c-3b55-432a-9aa9-e9908995811b"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_a558dbb8-cb91-4346-b7bc-c2a5dc5bae1a" width="907" height="342"></a> </p> <p>&nbsp;</p> <p>Save the Step.</p> <p>&nbsp;</p> <p><strong>Add a new ‘FTP Upload’ Step</strong></p> <p>We have assumed the there is a corresponding file called medals.csv in a local file system that we would like to upload to the FTP server.</p> <p>&nbsp;</p> <p>Add a ‘Ftp File Upload’ step and name it ‘Upload File via FTP (s)’ and then configure it.</p> <p>&nbsp;</p> <p><strong>Source Filename </strong> <p>This is the name and path of the file to be uploaded into the FTP Server eg <em>c:\temp\local\medals.csv</em> <p>&nbsp; <p><strong>FTP Server Path</strong>&nbsp; <p>This is the url style path of the of the location you want the file uploaded to eg <a title="ftp://localhost/" href="ftp://localhost/"><em>ftp://localhost/</em></a> <p>&nbsp; <p><strong>FTP Server uses Passive Mode</strong> <p>This allows the FTP server to change the port the ftp process is working through. <em>This is normally enabled .</em> <p><em></em>&nbsp; <p><strong>FTP Server uses SSL</strong> <p>If the server is configured to transfer files and authentication details over SSL then you will need to enable this setting.<em>This is normally enabled .</em> <p><em></em>&nbsp; <p><strong>FTP Server accept SSL certificate</strong> <p>It is possible (and likely) that the certificate used to secure the FTP server is not signed by a trusted root authority or is not trusted by your own machine. In this case you can indicate that it is OK to accept this SSL certificate. <em>This is optionally enabled .</em> <p>&nbsp; <p><strong>Delete Local File after Upload</strong> <p>After uploading you can choose to delete the source file. eg <strong>Disabled</strong> <p>&nbsp; <p><strong>Username</strong>&nbsp; <p>Set the username used to authenticate with the FTP server. <p>&nbsp; <p><strong>Password</strong> <p>Set the corresponding password to used with the Username <p>&nbsp; <p><strong>Timeout</strong> <p>This is the time in seconds that we will wait until giving up for a response from the FTP server. eg <em>Normally 30 seconds&nbsp; 00:00:30 is reasonable.</em></p> <p> <p>&nbsp;</p> <p>Once configured and Saved the step should look lie this:</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_1531b0d2-f8ee-4619-8a24-84a9f385c6a6"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_e777feb1-7ec4-4b1d-addc-4b677fb62ba7" width="907" height="342"></a> </p> <p>&nbsp;</p> <p>The can now download and upload via FTP with or without SSL transport enabled by running the project.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_816c5fd4-2dcb-4291-ae0a-827b1146583e"><img title="Upload and Download via FTP" style="display: inline" alt="Upload and Download via FTP" src="https://simegoimages.blob.core.windows.net/images/blog_e4968afb-75d2-49ba-88aa-732fc9399c61" width="1014" height="683"></a></p> Team Blog Tue, 22 Apr 2014 16:26:07 +0000 http://www.simego.com/Blog/2014/04/Speeding-up-small-updates-to-large-data-using-Incremental-synchronisation david@simego.com (David Evans) Speeding up small updates to large data using Incremental synchronisation <p>When we are integrating two different systems we often want to just utilise Simego DS3 to insert or update a small portion of the destinations data. For example we might want to synchronise 100 client records from a source file containing client records to our Dynamics CRM environment containing thousands of records. </p> <p>&nbsp;</p> <p>Using DS3’s normal compare mode, it would download the entire destination records, compare them with the source and present the results</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d1f4ae0e-a6d5-4977-a046-90fbdecaf208"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_b757ea23-8d8e-43a8-8899-1bc6d0f3766c" width="619" height="487"></a> </p> <p>&nbsp;</p> <p>In cases like this we have no interest that there are 24944 records in the destination that are not in the source. We are only interested in the 99 updates but have used 14 seconds to read the entire destination records. If we scale the problem it becomes more obviously significant when reading millions of records from the destination.</p> <p>&nbsp;</p> <p>In cases like these we can instruct DS3 to reconcile A to A incrementally. This means DS3 will only load the records found in A from B. It can identify that records are missing in B and if they have changed also. It will not detect if there are any to delete but we are not interested in deletion in this scenario. DS3 uses the primary keys in the source to request only the records in parallel from the target. This means that you may have only one primary key for this to work.</p> <p>&nbsp;</p> <p>To turn on Incremental mode for a project, navigate to the File menu then the project properties. Change the <strong>SyncOption</strong> from <strong>AtoB</strong> to <strong>AtoBIncremental.</strong></p> <p><strong></strong>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_fee416ce-78f5-4a66-9ac2-52c339ee7793"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_63900ec8-83a0-449b-9a1b-9503ea6bcfb8" width="389" height="465"></a> </p> <p>&nbsp;</p> <p>As you can see using the <strong>AtoBIncremental</strong> setting has reduced the load time from 14.147 seconds to 1.418 and only presented us with the updates rather than deletes.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ce50d306-3989-4c86-9fd7-a7d5ed70bb39"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_f95efeb7-98ba-4ab6-8117-91dbac4ad4a8" width="618" height="460"></a></p> Team Blog Mon, 21 Apr 2014 15:00:17 +0000 http://www.simego.com/Blog/2014/04/Triggering-a-Synchronisation-Project-from-Dynamics-CRM-Online david@simego.com (David Evans) Triggering a Synchronisation Project from Dynamics CRM Online <p>There are many ways to integrate data between systems. What is often forgotten is that we only want to process these imports or exports when something has changed.</p> <p>&nbsp;</p> <p>We often just run the synchronisation or integration at a specific time or periodically.</p> <p>&nbsp;</p> <p>For example we may want to identify when a new contact has been created or updated within Dynamic CRM Online. We perhaps want this information to be synchronised with a local SQL table for reporting purposes.</p> <p>&nbsp;</p> <p>We currently have several choices.</p> <p>&nbsp;</p> <p><strong>Synchronise on a regular basis</strong>. We can just reconcile and then synchronise on regular interval between the Dynamics CRM contact list to our local database contacts table. This technique requires that each time we connect to Dynamics CRM, we authenticate, then query the Contacts entity to evaluate if there has been any changes. This is obviously increases the workload on Dynamics CRM Online and potentially 95% of the time there will be no data changes&nbsp; and therefore no synchronisation required. </p> <p>&nbsp;</p> <p><strong>Poll continuously looking to see if there are any new records or records that have changed</strong>. This technique requires that each time we connect to Dynamics CRM, we authenticate, then query the Contacts to evaluate if there has been any changes. If there are changes then request a full synchronisation. This method is slightly less onerous that just synchronising regularly but is still impactful on the Dynamics CRM Online workload.</p> <p>&nbsp;</p> <p><strong>Send message via the Azure Message Bus</strong>. This seems ideal as Microsoft has given us some infrastructure to pass messages by. Unfortunately, we will have to go through some significant authentication and authorisation steps.&nbsp; In addition we would want something flexible enough to attach to any changes which we have to program ourselves. This would be technically challenging and expensive to develop.</p> <p>&nbsp;</p> <p><strong>Create a Plug-in or Custom Workflow.</strong> We can use the built in Dynamics CRM expandability to create a plug-in or better a workflow step that will notify the synchronisation engine that an event has occurred and it should begin synchronisation. The challenge in this scenario is that we will need to create some form of authentication to allow us to tell the on-premise system that there have been changes. In addition we would need to configure some security from the internet based CRM Online to the on-Premise synchronisation engine, which may be behind a firewall.</p> <p>&nbsp;</p> <p><strong>Design</strong></p> <p>What we really need is a combination that requires no authentication, no development and is easy to configure?</p> <p>&nbsp;</p> <p>1. Create a workflow component we can call from anywhere within Dynamics CRM that can notify an external system when something has changed.</p> <p>&nbsp;</p> <p>2.Create an internet notification store that can be told when something has changed and can be queried about those changes. No identifying data to be stored as no authentication required because we don’t want to store security credentials in the Dynamics CRM environment or tunnel through internal firewalls.</p> <p>&nbsp;</p> <p>3. Create a polling system that can monitor the internet based store for changes and trigger processes if there has been a change.</p> <p>&nbsp;</p> <p><strong>The Dynamics CRM Workflow Component</strong></p> <p>You can download it from here (including source code).</p> <p>&nbsp;</p> <p><a title="http://www.simego.com/downloads/Simego.CRM.Workflow.zip" href="http://www.simego.com/downloads/Simego.CRM.Workflow.zip">http://www.simego.com/downloads/Simego.CRM.Workflow.zip</a></p> <p>&nbsp;</p> <p>Register the component the same as any other Dynamics CRM workflow/plug-in ( Simego.CRM.Workflow.dll).</p> <p>&nbsp;</p> <p>Create a process and call the workflow component</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_4b207149-79d2-43b8-8deb-fd79ddfc2e70"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_047b4273-374d-4d1d-bae5-0af99b0671ea" width="796" height="709"></a> </p> <p>&nbsp;</p> <p>Configure the workflow step to use a made up GUID. I chose this one but you can use anyone BUT remember it as we have to tell the monitoring system to watch for it. </p> <p>&nbsp;</p> <p>a8243db5-fbf7-4637-881e-d635ca99269c</p> <p>&nbsp;</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_df1eb804-aeba-437f-b670-dd42d403ca00"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_22928aa3-80b9-4f50-9c23-57c9e9f0f5ee" width="847" height="267"></a> </p> <p>&nbsp;</p> <p></p> <p>Save and Close then Activate the workflow.</p> <p><strong></strong>&nbsp;</p> <p><strong>The ‘Internet Notification Store’</strong> </p> <p><strong>Using the Simego Online Last Changed Service</strong></p> <p>Simego has developed a solution for exactly this purpose. It can be used by any system that can call a URL. If you want to read about in details the help is here:</p> <p>&nbsp;</p><pre class="csharpcode">http://www.simego.com/Help/Online/Last-Change-Service</pre> <style type="text/css">.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } </style> <p>&nbsp;</p> <p>Every time a Contact now changes the workflow is going to call this URL:</p> <p>&nbsp;</p><pre class="csharpcode"><a href="https://online.simego.com/Change/Update/a8243db5-fbf7-4637-881e-d635ca99269c">https://online.simego.com/Change/Update/a8243db5-fbf7-4637-881e-d635ca99269c</a></pre> <style type="text/css">.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } </style> <p>&nbsp;</p> <p>and when that URL is called it will update the last ‘lastchanged’ date time associated with the GUID.</p> <p>&nbsp;</p> <p>This allows any system to query the URL and find out if something has changed since the last time it looked. This is unsecured but contains no identifying information and scales incredibly well. </p> <p>&nbsp;</p> <p>To query the last change we just call this URL:</p> <p>&nbsp;</p><pre class="csharpcode"><a href="https://online.simego.com/Change/a8243db5-fbf7-4637-881e-d635ca99269c">https://online.simego.com/Change/a8243db5-fbf7-4637-881e-d635ca99269c</a></pre> <style type="text/css">.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } </style> <p><strong></strong>&nbsp;</p> <p><strong>The ‘Internet Notification Store’ Polling</strong></p> <p>&nbsp;</p> <p>Simego DS3 Automation Server and Simego Online both have the ability to poll the ‘Last Changed Service’ and Trigger associated projects by using a JSON Trigger.</p> <p>&nbsp;</p> <p><strong>Attaching a Trigger using the Json Trigger in the Automation Server.</strong> </p> <p>&nbsp;</p> <p>Open the automation (Ouvvi) server and navigate to the triggers page. Choose Add Trigger and select the <strong>Json Trigger.</strong></p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_c53c50f3-31a0-4643-b606-258c57790d3a"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_4d4ed314-b0cf-4947-b39c-cd0aedfb1ebb" width="627" height="480"></a> </p> <p>&nbsp;</p> <p>Configure the trigger with your Service URL </p> <p>&nbsp;</p><pre class="csharpcode"><a href="https://online.simego.com/Change/a8243db5-fbf7-4637-881e-d635ca99269c">https://online.simego.com/Change/a8243db5-fbf7-4637-881e-d635ca99269c</a></pre> <p>&nbsp;</p> <p>Set the Json Expression to ‘lastchange’. </p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_22f22716-0afb-4087-be0f-b85741b3ba25"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_90bd72e0-4e40-4459-9252-f73cfaf224d5" width="863" height="768"></a> </p> <p>&nbsp;</p> <p>We can now create a synchronisation project to synchronise data from Dynamics CRM Online to our on-premise database.</p> <p>&nbsp;</p> <p>Every time there is a change in the contact entity online it will be synchronised locally.</p> Team Blog Wed, 16 Apr 2014 13:51:56 +0000 http://www.simego.com/Blog/2014/04/Web-API-Ouvvi-Step-Handler info@simego.com (Sean Cleaver) Web API Ouvvi Step Handler <p>We have migrated the really handy Web API Step Handler from Simego Online into DS3 Automation Server (Ouvvi) this allows you to setup calls to Web Applications that have documented API’s. such as other notification systems.</p> <p>&nbsp;</p> <p>For example configuring the Web API Step to send messages to your mobile devices based on the <a href="https://pushover.net/">Pushover</a> system is easy as setting up the request like this below.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_0da656cb-0299-4685-9529-81105d508f64"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_8677b1bd-e2f2-476a-b1ec-b7f931dacb63" width="892" height="756"></a> </p> <p>&nbsp;</p> <p>&nbsp;</p> <p>You can also use this to Test your Web API’s and ensure that the API’s are working as intended. Or perhaps to use this as a Queue Relay Broker service to integrate application messages.</p> <p>&nbsp;</p> <p>The Web API allows you to control the request details and validate the response so you can test to ensure the response is what you expected.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ccce2684-d1ba-4889-8a09-d18fe2b1376a"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_8f063ca9-5e31-4325-9d25-b09edda6bb95" width="602" height="768"></a></p> Team Blog Tue, 15 Apr 2014 14:29:33 +0000 http://www.simego.com/Blog/2014/04/Ouvvi-32-64-bit-Data-Sync-Options info@simego.com (Sean Cleaver) Ouvvi 32/64 bit Data Sync Options <p>We have a new release of both Data Sync and DS3 Automaton Server (Ouvvi) that enable the ability to run both 64-Bit and 32-Bit processes from a single installation.</p> <p>&nbsp;</p> <p>The 64 Bit version of Data Sync now ships with 32 bit components as well, and from Ouvvi you can choose how to run the Data Sync Task. This can be either <strong>In-Process</strong> (Default) this is how it works today, and we have two new options <strong>External 32-Bit Process</strong> and <strong>External 64-Bit Process</strong>. </p> <p>&nbsp;</p> <p>These new options run the Data Sync Task in a new Process that is either 32 Bit or 64 Bit, this enables you to mix sync tasks when you need to access 32 bit Legacy ODBC Drivers whilst maintaining an 64 Bit Installation.</p> <p>&nbsp;</p> <p>To set the execution mode you simply select the mode of operation on the Data Sync Task in Ouvvi.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_4118a220-5da3-4add-b926-5710716e63ca"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_c5ec2eb6-5028-4e07-a535-ca866108a541" width="854" height="750"></a></p> Team Blog Tue, 15 Apr 2014 13:15:13 +0000 http://www.simego.com/Blog/2014/04/Connection-Library-Data-Preview info@simego.com (Sean Cleaver) Connection Library Data Preview <p>We&rsquo;ve been busy extending the Connection Library features of data sync to really help you discover your data whilst building out your migration or integration.</p> <p>&nbsp;</p> <p>You can now preview data right from the context menu in the Connection Library this works against any data object in the connection library and will return approximately the first 1000 items.</p> <p>&nbsp;</p> <p>For Dynamics CRM we also now support <strong><a href="/Help/DataSync/Lookup-CRM-OptionSet-Values">OptionSets</a></strong> directly which can be used for drag &amp; drop Lookups or on their own.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_a7d96f53-c313-4c7d-93ba-98abb548e776"><img height="480" width="574" src="https://simegoimages.blob.core.windows.net/images/blog_1869746e-ff53-4a0c-ba0c-28c910a52212" alt="image" border="0" style="display: inline; border: 0px;" title="image" /></a></p> Team Blog Fri, 11 Apr 2014 10:23:38 +0000 http://www.simego.com/Blog/2014/03/Using-FTP-and-Simego-Automation-Server-to-Import-and-Export-files david@simego.com (David Evans) Using FTP and Simego Automation Server to Import and Export files <p>FTP continues to be a highly used internet protocol despite many innovative products like Dropbox, SkyDrive amd Box.net. Unfortunately, due to some considerable complexity in implementing downloading and uploading FTP, developing your own solutions can be painful.</p> <p>&nbsp;</p> <p>We have worked with our clients to provide the easiest and most consistently successful method of downloading and uploading files to FTP using the power and auditability of Simego Automation server and&nbsp; ‘Wget’, a well know and reliable HTTP. HTTPS and FTP command line application.</p> <p>&nbsp;</p> <p>1. Download the Wget Binaries and place them in a location on the Automation Server (Ouvvi).(C:\Temp\FTPClient)</p> <p>&nbsp;</p> <p><a title="https://www.gnu.org/software/wget/" href="https://www.gnu.org/software/wget/">https://www.gnu.org/software/wget/</a></p> <p>&nbsp;</p> <p>Download the binaries here: <a title="http://gnuwin32.sourceforge.net/packages/wget.htm" href="http://gnuwin32.sourceforge.net/packages/wget.htm">http://gnuwin32.sourceforge.net/packages/wget.htm</a> in the bin directory is the wget.exe which needs to be copied.</p> <p>&nbsp;</p> <p>2. Create 2 new 'User Settings' in Ouvvi<br>Settings -&gt; User Settings -&gt; Add Setting</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_36107972-69b9-4af9-ba87-fd46f1be1e08"><img title="ConfigSetting1" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="ConfigSetting1" src="https://simegoimages.blob.core.windows.net/images/blog_f7f9cd0e-b024-474d-b47f-8b4c844c6d8e" width="942" height="137"></a> </p> <p><br>3. Create a new 'External Program' step in a project to call the wget command line tool download the files .</p> <p></p> <p></p> <p></p><a href="https://simegoimages.blob.core.windows.net/images/blog_930487a0-b87b-4aef-9c05-217ad105c6d7"><img title="ConfigSetting2" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="ConfigSetting2" src="https://simegoimages.blob.core.windows.net/images/blog_26c21f1f-ba10-424c-a828-60d1a25b9a6e" width="972" height="252"></a> <p>&nbsp;</p> <p>Running the above will download the i386.exe file to the working directory.</p> <p>&nbsp;</p> <p>You can browse all of the functionality of wget here <a title="https://www.gnu.org/software/wget/manual/" href="https://www.gnu.org/software/wget/manual/">https://www.gnu.org/software/wget/manual/</a> which includes an excellent way to synchronise FTP and local directories</p> Team Blog Mon, 24 Mar 2014 12:59:32 +0000 http://www.simego.com/Blog/2014/03/Simple-Application-Integration-with-Simego-Online info@simego.com (Sean Cleaver) Simple Application Integration with Simego Online <p>This is a quick example of how we used Simego Online Platform to quickly provide some application integration between our systems without coding. (Implementation time 30 minutes)</p> <p>&nbsp;</p> <p>We started out like a lot of people were we had a website then a LOB application (Helpdesk) both systems were independent with their own Database, list of users all hosted on Windows Azure. This was fine for a while but then it got to a point where it would be nice to at least link the user profiles.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_47c8a87b-880b-4e2a-a960-3a273a336e2c"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; float: none; padding-top: 0px; padding-left: 0px; margin-left: auto; border-left: 0px; display: block; padding-right: 0px; margin-right: auto" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_b38db1b3-d7dd-46ba-9cbc-f9ba4fa73d88" width="462" height="369"></a></p> <p>&nbsp;</p> <p>Simego Online and Data Sync to the Rescue, by setting up a simple Simego Online project to run a Data Sync project based on users updating their profile on our main website, profiles in our HelpDesk are kept in Sync.</p> <p>&nbsp;</p> <p>We used the SQL (Azure) Trigger in Simego Online to run the project when ever a change was made so that changes were reflected in near real-time. </p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_33f1b560-ddef-4251-8d72-37a032e06455"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; float: none; padding-top: 0px; padding-left: 0px; margin-left: auto; border-left: 0px; display: block; padding-right: 0px; margin-right: auto" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_2134b46a-3edd-4fb6-8ffa-66e12051714b" width="559" height="384"></a></p> <p>&nbsp;</p> <p>The benefits here are this is all managed in the Cloud on the Simego Online platform, so we do not need to manage any servers, databases On Premise and the whole process just works!</p> <p>&nbsp;</p> <p>We could so easily now add another project to sync user profile data into Dynamics CRM or other systems all automatically.</p> Team Blog Mon, 17 Mar 2014 12:07:52 +0000 http://www.simego.com/Blog/2014/03/Application-Asynchronous-Integration info@simego.com (Sean Cleaver) Application Asynchronous Integration <p>By Leveraging Simego Online platform you can use Simego Online to call your application back to create asynchronous tasks. You might want to leverage this capability for long running tasks where you do not want to build background services but rather just create simple HTTP Services that are called.</p> <p>&nbsp;</p> <p>For example, simple tasks like sending email, querying mailboxes can take some time to execute where they could be executed outside of the user context.</p> <p>&nbsp;</p> <p>We use this service to run the background processing of our HelpDesk without the need to run a separate server. Our HelpDesk runs on Windows Azure Websites and all the background processing is handled via callbacks from Simego Online.</p> <p>&nbsp;</p> <p>One of our process flows is as follows</p> <p>New Email &ndash;&gt; Call HelpDesk to Process Inbox &ndash;&gt; For Each Message Set up Callback to get Message and Process</p> <p>&nbsp;</p> <p>Then we move onto</p> <p>Get Message &ndash;&gt; Add to HelpDesk &ndash;&gt; Set up Callback to Email User</p> <p>&nbsp;</p> <p>This means we can create very small ASP.NET MVC Controller Actions that execute a simple task and tie them together via Simego Online Callbacks. All this then runs in the Cloud as if by magic and we can view every execution form Simego Online and even re-submit failed tasks.</p> <p>&nbsp;</p> <p>Example dashboard showing HelpDesk operations being executed against our HelpDesk Website.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_84465bc5-ce28-4397-b2e4-7f0afb017e18"><img height="416" width="1040" src="https://simegoimages.blob.core.windows.net/images/blog_996d85b4-f77c-4e05-856c-25024270d4c9" alt="image" border="0" style="background-image: none; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border: 0px;" title="image" /></a></p> <p>&nbsp;</p> <p>So how do you set this up?</p> <p>&nbsp;</p> <p>Simego Online has a very useful Step Handler called Web API this allows you to execute Web Actions and validate their responses. This could be used for Web API End Point testing or anything else where you want to call an API endpoint.</p> <p>&nbsp;</p> <p>You can send GET/POST/PUT/DELETE requests, add Headers, add Body, Basic Authentication and validate the response.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_fe694dd7-c980-4687-bd5a-0067710d90f6"><img height="741" width="814" src="https://simegoimages.blob.core.windows.net/images/blog_50531bcc-d7b5-4805-b8fc-de93e3adb42a" alt="image" border="0" style="background-image: none; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border: 0px;" title="image" /></a></p> <p>So in our HelpDesk one of our useful projects is a simple Relay service where we send in a message that gets relayed back to us some time later for processing. Because we don&rsquo;t want to get overloaded this all happens in a nice orderly fashion 1 at a time. We also have the ability to configure a retry and failure condition. We could also setup an Email Task to let us know if something fails.</p> <p>&nbsp;</p> <p>We&rsquo;re also using a little know feature of Simego Online that allows us to expand variables at runtime, so in the configuration of the step we set the URL to <strong>{{ReturnURL}}</strong> which is actually a value passed to Simego Online by the Client Application, we also use this in the Body so when we get called back we receive an ID value that means something to us. We also validate that the Response returns a 200 OK otherwise we retry 3 times before putting the message on the Failure Queue.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_c702ddeb-d1b6-4a8d-a365-2ec42ff795cd"><img height="412" width="640" src="https://simegoimages.blob.core.windows.net/images/blog_49a313db-fb59-464f-9b3a-46263f225dad" alt="image" border="0" style="background-image: none; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border: 0px;" title="image" /></a></p> <p>&nbsp;</p> <p>To initiate a Task to run we need to POST our message data to Simego Online to start the Project.</p> <p>&nbsp;</p> <p>The URL is like this</p> <p>https://online.simego.com/&lt;account&gt;/&lt;workstream&gt;/ProjectApi/Start/&lt;ProjectName&gt;</p> <p>&nbsp;</p> <p>Where you replace account, work stream and project name with your values.</p> <p>&nbsp;</p> <p>The Body you Post is a Json value with the details you want to use in the project. This URL is basic auth over SSL authenticated so you should create an API User account under Settings &ndash;&gt; User Profiles and set the Permission Level to User so that this user can start projects.</p> <p>&nbsp;</p> <p>We&rsquo;re using Return URL and ID in our Step so our body is.</p> <p>&nbsp;</p> <p>{ ReturnURL : "MyURLGoesHere", ID: 1234 }</p> <p>&nbsp;</p> <p>Then when Simgo Online executes the Project we get called back via a POST to our URL with the ID value then it&rsquo;s up to us to simply execute the Task.</p> <p>&nbsp;</p> <p>We setup a simple Helper Function to Post the Message to Simego Online like this.</p> <p>&nbsp;</p> <p>public string PostJsonRequest(string url, string jsonString)<br />{<br />&nbsp;&nbsp;&nbsp; HttpWebRequest webRequest = (HttpWebRequest)HttpWebRequest.Create(url);<br />&nbsp;&nbsp;&nbsp; webRequest.Method = "POST";<br />&nbsp;&nbsp;&nbsp; webRequest.ContentType = "application/json";<br />&nbsp;&nbsp;&nbsp; webRequest.Accept = "application/json";<br />&nbsp;&nbsp;&nbsp; webRequest.ServicePoint.Expect100Continue = false;<br />&nbsp;&nbsp;&nbsp; webRequest.Credentials = new NetworkCredential(Settings.GetSetting&lt;string&gt;(SettingKey.QueueApiUsername), "");</p> <p>&nbsp;&nbsp;&nbsp; if (!string.IsNullOrEmpty(jsonString))<br />&nbsp;&nbsp;&nbsp; {<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; byte[] data = Encoding.UTF8.GetBytes(jsonString);</p> <p>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; webRequest.ContentLength = data.Length;</p> <p>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; using (Stream requestStream = webRequest.GetRequestStream())<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; {<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; requestStream.Write(data, 0, data.Length);<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }<br />&nbsp;&nbsp;&nbsp; }</p> <p>&nbsp;&nbsp;&nbsp; using (HttpWebResponse response = (HttpWebResponse)webRequest.GetResponse())<br />&nbsp;&nbsp;&nbsp; {<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; using (StreamReader sr = new StreamReader(response.GetResponseStream()))<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; {<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; return sr.ReadToEnd();<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }<br />&nbsp;&nbsp;&nbsp; }<br />}</p> <p>&nbsp;</p> <p>This system allows us to easily deploy new services that execute background tasks via simple web sites that otherwise would have become complicated and require multiple services.</p> <p>&nbsp;</p> <p>You can of course simply use this to be called on a schedule like every 5 minutes to run some background tasks.</p> Team Blog Mon, 17 Mar 2014 11:35:32 +0000 http://www.simego.com/Blog/2014/02/Announcing-Intercom-Data-Provider-for-DS3 info@simego.com (Sean Cleaver) Announcing Intercom Data Provider for DS3 <p>We have a new Data Provider available for <a href="http://www.intercom.io">Intercom</a>, which provides full Read/Write Capability. Allowing you to Import, Export and Synchronise data between many different systems and Intercom. </p> <p>&nbsp;</p> <p>For example with this you could set up an Intercom to Dynamics CRM Lead Capture integration, or use Data Sync to decorate your Intercom data with data from your own internal databases.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_74e78f36-5826-4c8f-aed0-3b8e8ae62a23"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_834b51ad-69c5-4a35-af82-bb32cbb1a696" width="910" height="501"></a></p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_3ec5b9c9-6cf4-4d1e-95e9-60495738d813"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_8a20c32a-bc06-4107-b8c6-43e866bbadbf" width="625" height="480"></a></p> <p>&nbsp;</p> <p>The Full Source Code for the Intercom Provider is available in the Data Sync Installation Directory so that you can study it to build similar REST API providers or to adapt it to your own needs.</p> Team Blog Thu, 20 Feb 2014 15:47:06 +0000 http://www.simego.com/Blog/2014/02/Creating-Custom-Data-Providers-with-VS info@simego.com (Sean Cleaver) Creating Custom Data Providers with Visual Studio <p>We’ve added a new feature to make it super easy to start building your own data providers for Data Sync. This feature is currently in beta so you need release 3.0.808 or greater.</p> <p>&nbsp;</p> <p>The feature from a Click of a Button creates a Visual Studio Project with a Template Project that can be opened in Visual Studio and give you F5 run and Debug.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f55d09d0-9ca4-4b4c-8c57-3c94eb973fcc"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_433e8e2f-9044-4abf-8d3d-a0ddd074943e" width="470" height="589"></a></p> <p>&nbsp;</p> <p>&nbsp;</p> <p>&nbsp;</p> <p>This new Project Opened in Visual Studio 2012</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_c1ab4b69-c260-4855-935d-11a5c00c123f"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_83741756-d2d5-47a5-8795-8866ed509744" width="999" height="768"></a></p> <p>&nbsp;</p> <p>Full F5 Debugging via Visual Studio to help you build the provider.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_6dc89065-db1c-4f7b-81bd-2647b58299b1"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_126345f9-b8d1-4b7c-804d-688509c0164a" width="1278" height="982"></a></p> <p>&nbsp;</p> <p>This allows you to code up whatever you want and then deploy the assembly to Data Sync and use it just like the providers we have built.</p> <p>&nbsp;</p> <p>We often get asked how long does it take to write a provider for Data Sync. It really depends on the system you connect to and the API it exposes, however you can generally be up and running with a simple read-only provider in a couple of hours. We recently created a complete read-write provider for <a href="http://intercom.io/">Intercom</a> which is a JSON REST based API and this took approximately 8 hours over a weekend to complete.</p> Team Blog Wed, 19 Feb 2014 17:41:29 +0000 http://www.simego.com/Blog/2014/02/New-Connection-Library-Video info@simego.com (Sean Cleaver) New Connection Library Video <p>Quick video showing how easy it is now to connect to your source or target data, preview the data and attach a Lookup to the data source.</p> <p>&nbsp;</p><iframe height="360" src="http://feeds.simego.com//www.youtube.com/embed/pKK1GAe8YKo?rel=0" frameborder="0" width="640" allowfullscreen></iframe> Team Blog Thu, 06 Feb 2014 16:15:46 +0000 http://www.simego.com/Blog/2014/02/Data-Sync-Connection-Library-Update info@simego.com (Sean Cleaver) Data Sync Connection Library Update <p>We’re getting ready to release a major productivity boost for Data Synchronisation Studio. It will now be possible to store connection details to your systems and then use the NEW Connection Library schema explorer to connect to your data sources. </p> <p>&nbsp;</p> <p>You simply browse to the object you want to connect to and just drag and drop the object onto the connection source or target and that’s it!</p> <p>&nbsp;</p> <p>You can even create lookups by just dragging the related object to the Column in the Datasource to create the Lookup connection.</p> <p>&nbsp;</p> <p>The connection Library explorer will support all the major providers, SQL Server, ODBC, OleDB, SharePoint, Dynamics CRM, Podio, Active Directory and Simego Online WebStore.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f32de88f-c4a4-44d7-84fe-a575fb6c7f13"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_4eba35b7-3cb1-4b67-98af-c155f91653e6" width="283" height="1257"></a></p> Team Blog Mon, 03 Feb 2014 14:20:31 +0000 http://www.simego.com/Blog/2014/01/Integration-via-Published-API david@simego.com (David Evans) The Next step in B2B integration <p>At Simego we have been thinking and working hard with our clients to utilise emerging web based API standards to enable simple B2B and departmental integration. At its core is Simego WebTable and surrounding it is a feature complete cloud data and integration environment.</p> <p>&nbsp;</p> <h4>What will B2B look like using Simego Online and Web Table</h4> <p>Essentially, your business analyst selects a set of data from a system or file and publishes to your clients using modern web standard and security.</p> <p>Using a simple a Desktop Application (Simego DS3), the analyst choose the source and creates a new Web Table.</p> <p>&nbsp;</p> <p>The Simego WebTable exposes the data in a modern web API, which is accessible through all of the emerging standards. In addition Simego WebTable makes the data accessible through excel and the client can even use Simego DS3 to integrate their WebTable data with other systems.</p> <p>&nbsp;</p> <h5>An Example</h5> <p>Hedge Fund A has several clients, all of whom want access to &lsquo;their data&rsquo; in as real time as possible, in this case trades. Classically their options would be to create a complex integration involving FTP sites, servers, certificates which would be time consuming and risky. We would recommend using Simego Online and a &lsquo;Web Table&rsquo; to synchronise client data in near real-time. This would allow each client to choose what format they would like to download the data and you could choose the level of security.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_22c6adec-5767-49bc-85ea-cd4a01a01b9d"><img height="655" width="1460" src="https://simegoimages.blob.core.windows.net/images/blog_e33834b2-ad36-411b-a5d6-c7688c849170" alt="image" border="0" style="display: inline; border: 0px;" title="image" /></a></p> <p>&nbsp;</p> <p>Using Web Table they could quickly give their client&rsquo;s access to their trades create a strong and unique relationship:</p> <p>&nbsp;</p> <p>https://online.simego.com/simegohedgefund/clienta/trades.csv</p> <p>https://online.simego.com/simegohedgefund/clienta/trades.xml</p> <p>https://online.simego.com/simegohedgefund/clienta/trades.json</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_75e22237-b440-4033-9f57-1f80fa7cbbb0"><img height="480" width="563" src="https://simegoimages.blob.core.windows.net/images/blog_5af20dd9-9c8c-41b5-a009-aad48bf86fc5" alt="image" border="0" style="display: inline; border: 0px;" title="image" /></a></p> <p>Each client relationship contained in a separate container:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_a50c9f38-2e00-42cb-b182-c8803a27193d"><img height="528" width="988" src="https://simegoimages.blob.core.windows.net/images/blog_95dece80-fa25-4d75-a900-ba471a8a4038" alt="image" border="0" style="display: inline; border: 0px;" title="image" /></a></p> <p>&nbsp;</p> <p>The data con be synchronised up to the client trades Web Table:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_da95abfa-b266-4b17-9ffc-ff4e27a0037b"><img height="1146" width="1320" src="https://simegoimages.blob.core.windows.net/images/blog_3e2df0c5-3e98-42fc-8f72-ac97f4fdb5cd" alt="image" border="0" style="display: inline; border: 0px;" title="image" /></a></p> <p>&nbsp;</p> <p>Separate cloud based projects can be set up to synchronise just the client portion of data to their</p> <p>trades web table. Each project has its own details audit and change control:</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_0c191086-b847-492a-86fb-2f67c022f513"><img height="768" width="921" src="https://simegoimages.blob.core.windows.net/images/blog_8f5e581e-f3dd-4a36-8a3d-957e91d9a928" alt="image" border="0" style="display: inline; border: 0px;" title="image" /></a></p> <p>&nbsp;</p> <p>This project moves only the specific data relating to that client to their Web Table. The move is triggered by any change in the source data. This makes the updates to the data available near real-time. Alternatively, a synchronisation can be scheduled periodically. In addition the system can send notification emails when a synchronisation is finished.</p> <h4>Reporting</h4> <p>To be able to manage any integration it is critical to have transparent reporting, clear dashboards and deep audit details.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_416e8026-c996-47f1-adc7-a635fab4ab59"><img height="768" width="921" src="https://simegoimages.blob.core.windows.net/images/blog_9c1c0f8a-7e58-484a-a3f4-b1cb36ba6a85" alt="image" border="0" style="display: inline; border: 0px;" title="image" /></a></p> <p>Once the data is synchronised with the clients WebTable, they can access it as if it was a download link or use it as an integration point for more advanced integration scenarios using xml, json, csv or as a source for DS3 to synchronise with internal client systems:</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_414c3df9-1ae9-4d05-8b7b-171e29a1b80a"><img height="650" width="1024" src="https://simegoimages.blob.core.windows.net/images/blog_6115f9a5-ec10-4968-86be-bf85aa89a8c5" alt="image" border="0" style="display: inline; border: 0px;" title="image" /></a></p> <p>If you would like a demonstration on how WebTable and Simego can make your business relationships unique and more profitable then please ask to see a demo or a trial invite from <a href="mailto:sales@simego.com">sales@simego.com</a>.</p> Team Blog Wed, 15 Jan 2014 13:31:19 +0000 http://www.simego.com/Blog/2014/01/Creating-a-Web-Store-Table info@simego.com (Sean Cleaver) Creating a Web Store Table <p>As part of Simego’s new Online services you can publish data online to share with your clients and partners. To create a Web Table is really easy with Data Synchronisation Studio.</p> <p>&nbsp;</p> <p>First connect to your Source Data in DS3.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_45c64607-e875-4fc3-a418-02a3e535228e"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_b809681e-bf3b-4d2e-b910-6730df7960a0" width="640" height="436"></a> </p> <p>&nbsp;</p> <p>Then from the Tools menu select Create Web Store List</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_078101da-bda7-4020-9c44-529d0f507791"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_21f1a979-b29f-4f82-8a51-2e89ace884b4" width="651" height="448"></a> </p> <p>&nbsp;</p> <p>Then enter the credentials to your Online Account, choose your Workstream and give your List a Name. (If you do not have a Workstream you will need to create a Workstream in your online account first.)</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_0d9eacf2-7920-4663-b9af-1e9094ccac07"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_232d6209-c58f-4112-b9eb-8edae1b04c34" width="495" height="328"></a>&nbsp;</p> <p>&nbsp;</p> <p>This will then create the structure of your List/Table in your Workstream under the Storage section.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d195d556-0836-486a-9312-879baeaeaefb"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_432b7f28-a95e-4280-80f0-64eb22b63095" width="640" height="190"></a> </p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_dec00e28-402b-4537-817f-abced5f50f48"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_895a1e88-2fad-4f88-aa72-70e7229c7e18" width="640" height="452"></a> </p> <p>&nbsp;</p> <p>Now you just need to Compare A-&gt;B and synchronise your Data into the Online List. Once this is done your Data is now available online in multiple formats. </p> <p>&nbsp;</p> <p>We currently support CSV, XML and JSON formats for the data depending on the URL you use.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_0932f115-812c-4e50-a7ba-e0c040cef6b0"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_614b6fb8-d040-44dd-b56d-979ef4e8b930" width="474" height="720"></a> </p> <p>&nbsp;</p> <p>You can also filter the data with simple URL parameters i.e. to filter records where Supplier ID = 1 we apply a filter parameter <strong>SupplierID eq 1</strong></p> <p>&nbsp;</p> <p>https://online.simego.com/&lt;SubscriptionName&gt;/&lt;WorkStreamName&gt;/StorageApi/List/Products.xml?filter=SupplierID%20eq%201</p> <p>&nbsp;</p> <p>Now we have a filtered view of the Data.</p> Team Blog Mon, 13 Jan 2014 13:40:05 +0000 http://www.simego.com/Blog/2014/01/Simego-Online-Service info@simego.com (Sean Cleaver) Simego Online Service <h6 class="mb">Today we&rsquo;re Announcing availability of Simego Online.</h6> <p class="mb">&nbsp;</p> <div class="mr-large"> <p>Simego Online is an advanced hybrid-cloud service for Data Integration allowing you to run simple and complex integrations in whatever location suits your business. You can choose whether to execute integrations on-premise or in the cloud from a single platform.</p> <p>&nbsp;</p> <p>Included in the new platform is the new Simego WebTable technology.&nbsp;Simego WebTable&nbsp;enables business to publish and consume data via standard methods such as CSV, XML and more advanced API publishing technology JSON using industry standard security methods using simple desktop applications and NO CODE.</p> <p>&nbsp;</p> <p>Both on-premise and cloud integrations can utilise these &lsquo;WebTables&rsquo; to create simple and powerful integration solutions enabling secure,&nbsp;near real-time&nbsp;and advanced B2B in minutes.</p> <p>&nbsp;</p> <p>You can choose where to run your workloads either OnPremise via our familiar products Data Synchronisation Studio and Simego Automation Server (Ouvvi) or In our Online Environment hosted in Microsoft Azure.</p> <p>&nbsp;</p> <h4>Packages</h4> <p>Enabling B2B solutions and internal integrations could not be easier. Quickly register with Simego Online and either evaluate the platform using a 'Trial' account or create a new package. Each package contains integration minutes, a DS3 Cloud Edition key for your business and the ability to create integration Workstreams.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_007243eb-bf90-4a6c-8937-1bfa994b6c80"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_d47f4093-202f-4492-acac-0b742027327b" alt="image" title="image" height="768" width="642" /></a></p> <p>&nbsp;</p> <h4>Data Synchronisation Studio &ndash; Online Edition</h4> <p>Data Synchronisation Studio &ndash; Online Edition is a version of the full DS3 client that integrates with the Online Platform.</p> <p>&nbsp;</p> <p>The Online Edition of DS3 links to your Online account and uses credit from your account each time you run a sync. Your can DS3 online license can be installed on multiple workstations and servers and used by multiple users.</p> <p>&nbsp;</p> <h4>Workstreams</h4> <p>Workstreams are the Online equivalent of Simego Automation Server (Ouvvi). Create a Workstream to host your integration projects and steps and run them in the cloud via our hosted environment. Create Schedules and trigger projects from external events or via HTTP REST API.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_655cb655-8683-4343-9bf0-86378bab1ca3"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_37a180c4-a5e6-4966-9e16-612d9daa3b05" alt="image" title="image" height="720" width="671" /></a></p> <p>&nbsp;</p> <h4>Web Table</h4> <p>Web Tables allow you to create lists of Data in the Cloud this is a great way to share data with third parties without having to stand up new servers, databases etc. Simply use DS3 online to publish your data to the Cloud and allow your clients to access the Data securely via JSON, XML or CSV end points.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_9e51130d-26b1-4b0d-9c30-281ffe05e289"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_40d2b781-fc05-418e-8413-92702e0f2c10" alt="image" title="image" height="768" width="715" /></a></p> <p>&nbsp;</p> <p>To find out more and for&nbsp;a demo account please contact our Team via our helpdesk<a href="mailto:support@simego.com">support@simego.com</a></p> </div> Team Blog Mon, 13 Jan 2014 11:45:33 +0000 http://www.simego.com/Blog/2013/11/Synchronise-a-SharePoint-Folder-with-the-File-System david@simego.com (David Evans) Synchronise a SharePoint Folder with the File System <p>Often it is necessary to keep a SharePoint Document Library synchronised with the local file system including sub-folders and Created Dates. This could not be easier using DS3!</p> <p>&nbsp;</p> <h5>Step 1. Connect the Source to a SharePoint Document Library</h5> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_0f3c7694-54fb-4d85-a971-3191c558ecb5"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_4690b23e-2f3e-47b2-8d0e-5fa44b05ce73" alt="image" style="display: inline; border: 0px;" title="image" height="1016" width="1460" /></a></p> &nbsp; <h5>Step 2. Connect the Destination to the file system</h5> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ba6ba05e-e31c-499c-808e-0ce487d85546"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_a70d67d6-7357-419a-9fb2-42fef7808c44" alt="image" style="display: inline; border: 0px;" title="image" height="867" width="1248" /></a></p> <p>&nbsp;</p> <h5>Step 3. Map the fields and Compare A -&gt; and Synchronise</h5> <p>Delete all other columns and create the following map:</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_966f9976-9e18-4c25-b13e-d8fba0a79dd5"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_44e39541-cfae-4458-abd1-8a2d609d7f4a" alt="image" style="display: inline; border: 0px;" title="image" height="114" width="690" /></a></p> <p>&nbsp;</p> <p>Click the Compare A &ndash;&gt; B button, then the Synchronise Button and then the Start Button and you will now have a SharePoint Document Library including subfolders and files on the file system.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d2f572ab-84cd-4e6e-92de-e8feaa78d432"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_9933de52-85d7-4388-af06-46becf8de7b6" alt="image" style="display: inline; border: 0px;" title="image" height="1022" width="1460" /></a></p> Team Blog Wed, 20 Nov 2013 09:48:11 +0000 http://www.simego.com/Blog/2013/11/How-to-download-Dynamics-CRM-Online-data david@simego.com (David Evans) How to download Dynamics CRM Online data <p>One of the challenges with Dynamics CRM Online, and the cloud in general, is integrating cloud data with all your internal touch points and requirements.</p> <p>&nbsp;</p> <p>If you want to do some detailed analysis of your account, contact or any entity data from Dynamics CRM, exporting a complete entity can take time and be laborious. Some companies like to have a &lsquo;backup&rsquo; in case of extended outages.</p> <p>&nbsp;</p> <p>Recently a client asked if he could download his 925000 row account table!</p> <p>&nbsp;</p> <h5>Step 1: Download and install the trial (you can currently do this using the trial edition up to 10k rows without registering!)</h5> <p><a href="http://www.simego.com/Install/Data-Synchronisation-Studio">http://www.simego.com/Install/Data-Synchronisation-Studio</a></p> <p>&nbsp;</p> <h5>Step 2: Open DS3 Developer and connect to Dynamics CRM Online</h5> <p>Click the green &lsquo;Connect Data Source&rsquo; on the left.</p> <p>Select the Dynamics CRM provider</p> <p>Enter in Dynamics CRM Online Details:</p> <p><span style="line-height: 20px;">&nbsp;</span></p> <ol> <li>Authentication: Federated</li> <li>URL: <a href="https://crm4.Dynamics.com">https://crm4.Dynamics.com</a></li> <li>Pick your organisation then the entity name.</li> </ol> <p>&nbsp;</p> <ul> <li><a href="https://simegoimages.blob.core.windows.net/images/blog_59c974a0-d02f-4e0a-83cb-4d2ac30fa346"><img height="721" width="1024" src="https://simegoimages.blob.core.windows.net/images/blog_ac33b8be-c5dd-40ec-9690-5529a6fda14b" alt="Connect to Dynamics CRM Online" border="0" style="display: inline; border: 0px;" title="Connect to Dynamics CRM Online" /></a>&nbsp;</li> </ul> <h5>Step 3: Select the columns we want to download</h5> <p>We must tell DS3 which columns we want to extract from the entity. The more columns we include, the more data that has to be downloaded which is therefore slower (Dynamics CRM will bring down about 1000 records per second). If you want ALL of the available columns, check the tick beside the &lsquo;Columns&rsquo; item in the columns tree view on the left.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f3520798-f508-454e-b6b9-7b9144e6a3cc"><img height="424" width="452" src="https://simegoimages.blob.core.windows.net/images/blog_52441154-5be3-47c5-8db4-db96db132b87" alt="Select the columns to download" border="0" style="display: inline; border: 0px;" title="Select the columns to download" /></a></p> <h5>Step 4: Use DS3 to create a CSV file for you.</h5> <p>In this step we use DS3 to create a CSV file with the same shape (columns) as your source entity.</p> <p>Click tools on the menu bar, then Select the &lsquo;Create CSV File&rsquo;. Enter in the destination of the CSV file.</p> <p>This creates a new CSV file and points the destination of the download to it.</p> <p>&nbsp;</p> <h5>Step 5: Compare and download the data</h5> <p>DS3 is more than just a download tool and is designed to synchronise two systems. In this case the CSV file is empty but we still have to compare it to the online entity.</p> <p>For performance you can change the Page Size in the CRM settings to be 5000 (the max permitted).</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_75a74d68-86dc-4544-8995-8129cbb70e1b"><img height="263" width="239" src="https://simegoimages.blob.core.windows.net/images/blog_4f0b235a-ef7f-460e-abd7-091fe202ea58" alt="image" border="0" style="display: inline; border: 0px;" title="image" /></a></p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_03d04328-ce38-4fa2-ad12-e12a99ad7d20"><img height="861" width="1222" src="https://simegoimages.blob.core.windows.net/images/blog_c64dd694-c187-470d-befc-1c18739d420f" alt="image" border="0" style="display: inline; border: 0px;" title="image" /></a></p> <p>1. Compare. Click the Compare A &ndash;&gt; B so that DS3 can determine what to add to the CSV File.</p> <p>DS3 will now download all of the data form the Dynamics Online Entity.</p> <p>2. Synchronise. DS3 will prepare the synchronisation and present you with the &lsquo;Synchronise Data&rsquo; Dialog.</p> <p>3. Start. Click the start button and DS3 will now write the data to the CSV files.</p> <p>&nbsp;</p> <p>When it has finished synchronizinga, all of the data will be written to the csv file.</p> <p>All your records will now be in the CSV files, for some more interesting downloads you can download to a SQL Table, an XML File , an XML Dataset, an Excel Sheet etc etc.</p> <p>&nbsp;</p> <p>This can be automated now using the DS3&rsquo;s Server component to just incrementally download the changes.</p> Team Blog Mon, 18 Nov 2013 11:24:33 +0000 http://www.simego.com/Blog/2013/11/Dynamics-CRM-Real-Time-Integration david@simego.com (David Evans) Dynamics CRM Real-Time Integration <p>It is more and more important to businesses to have up to date data to act upon. Customer service representatives informing their clients ‘If you wait a couple of minutes you should be able to log-in’ or ‘Your balance has not been updated yet, can I take a number or can you call back?’ are not what successful businesses are made of.</p> <p>&nbsp;</p> <p>Earlier on this year an Aberdeen Group report discovered that business leaders wanted more and more access to real-time data.</p> <p>&nbsp;</p> <p>David Linthicum, the author <a href="http://www.dataintegrationblog.com/data-integration/why-real-time-data-integration-is-hard-and-data-integration-technology-more-important-than-ever/" target="_blank">wrote</a> “The report noted that 89 percent of enterprises that use real-time integration have the power to provide managers with accurate information when it is needed, as opposed to only 73 percent of organizations that do not use real-time services,”</p> <p>&nbsp;</p> <p>This on face value looks like a 16% difference in capability (or competitive advantage). In reality, it is a huge difference due to the source of data no longer being from clean, managed data store but from core transactional systems.</p> <p>&nbsp;</p> <p>“This includes updating the real time data that is flowing to the decision maker, as the data changes over a given period of time.&nbsp; <strong>For instance, the ability to track factory production over an afternoon, as the production data changes minute-to-minute.</strong> <p><strong></strong>&nbsp; <p><strong>“In other words, we are moving from a report-oriented mentality to a dashboard-oriented mentality.”</strong> <p>&nbsp;</p> <h4>Who really needs Real-Time?</h4> <p>Finance and banking solutions are very reliant on systems being in-sync due to the timely nature of balances, pricing and transactions. Information such as available balances, current status of accounts and the actual cost of a transaction are required seconds after their change. As mentioned above, businesses who want to be able to build new processes should think hard of the benefits of not having to wait until later or worse an overnight update to make decisions.</p> <p>&nbsp;</p> <h4>What is involved in integrating in Real-Time?</h4> <p>There are obviously three critical stages to integrating data from one system into Dynamics CRM in ‘Real-Time’:</p> <ol> <li>Detection of the change in the source data. <li>Identifying if this change impacts on a Dynamics CRM destination entity. <li>Updating or Inserting data in the Dynamics CRM destination entity. </li></ol> <p>In addition there critical dimensions to the data changes:</p> <ol> <li>The amount of entities changed in the source entity (eg Start of Day balance updates) in a time window. <li>The time it takes to process an update or inserts vs the next required update window .(eg If there are 100 changes in 30 seconds and the integration solution takes 0.8 seconds to process a change serially then they system will never be in sync). </li></ol> <p>Critically Real-Time integrations have to be ‘in sync’ before starting the real-time data integration process. This is possible with preparation and good data management but the most difficult moment is when the source system or the Dynamics CRM system are unavailable. There must be a process by which once reconnected, the system will bring themselves back in sync. <strong>Recovery</strong> from unavailability, is the hardest problem in real-time synchronisation.</p> <p>&nbsp;</p> <h4>An example of Dynamics CRM Integration</h4> <p>As a demonstration I am going to configure a Real-Time synchronisation between a Microsoft SQL Server Table and a Dynamics CRM entity.</p> <p>&nbsp;</p> <p>For the example to be realistic we are going to test two scenarios and record the shortest and longest time from the change in the source systems to the update completing in the target system:</p> <ol> <li>Simulate a ‘Large Single Update’ such as a balance update in the morning or mass change in data. Updating 3000 records in the SQL Table a Dynamics CRM entity with 6000 items. <li>Simulate a ‘Regular Change’ of data in the source system: 5 updates every 5 Seconds from a table with 6000 records targeting a Dynamics CRM entity with 6000 items . </li></ol> <p><strong>&nbsp;</strong>&nbsp;</p> <p><strong>Detection of the change in the source data</strong></p> <p>To detect the change in the source table I attached a DS3 Monitoring Trigger (this is not a SQL ‘Trigger’ attached to the table). This monitoring trigger activates a DS3 Integration project when it detects a insert or update to the source table. There are many Monitoring triggers built into DS3 Server (File Monitor, SharePoint List, Http result etc etc). This should table about 1 minute to set up .</p> <p>&nbsp;</p> <p><strong>Identifying if this change impacts on a Dynamics CRM destination entity.</strong></p> <p>Create a new DS3 project mapping between the source SQL table and the target entity.</p> <p>&nbsp;</p> <p><strong>Updating or Inserting data in the Dynamics CRM destination entity.</strong></p> <p>I uploaded the DS3 project into DS3 Server. I can now connect the Trigger to the project.</p> <p>&nbsp;</p> <p><strong>Recovery</strong></p> <p>We don’t have to worry about recovery in this example as DS3 detects the last successful sync and automatically synchronises from there. This technique is only possible if you have a high performance insert/update process as you have to ‘catch-up’ i.e. Insert/update 100x faster that required to keep the system in sync.</p> <h4>Results</h4> <p>&nbsp;</p> <p><strong>Scenario 1: 6000 Instant Updates</strong></p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_af01fa06-b16a-4633-b9b9-6f07b8ee23d7"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_d61994d5-1dc9-4c67-852e-67d10c913fe9" width="676" height="341"></a></p> <p>Actual Timings</p> <table cellspacing="0" cellpadding="2" width="823" border="0"> <tbody> <tr> <td valign="top" width="354">Source SQL Table Updated</td> <td valign="top" width="93">16:11:23</td> <td valign="top" width="374">Start</td></tr> <tr> <td valign="top" width="354">First Update Appears in Dynamics CRM</td> <td valign="top" width="93">16:11:25</td> <td valign="top" width="374">3 Seconds from Start</td></tr> <tr> <td valign="top" width="354">Last Update Completes in Dynamics CRM</td> <td valign="top" width="93">16:11:42</td> <td valign="top" width="374">9 Seconds from Start</td></tr></tbody></table> <p>&nbsp;</p> <p><strong>Scenario 2: 5 Updates every 5 Seconds</strong></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_43111463-c931-4d57-9ba7-8df40f1a765a"><img title="image" style="display: inline" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_dc6f7f73-da45-4e3d-b006-79706264775b" width="676" height="341"></a></p> <p>&nbsp;</p> <p>&nbsp;</p> <p>Actual Timings</p> <table cellspacing="0" cellpadding="2" width="823" border="0"> <tbody> <tr> <td valign="top" width="350">Source SQL Table Updated</td> <td valign="top" width="108">16:51:18</td> <td valign="top" width="363">Start</td></tr> <tr> <td valign="top" width="350">First Update Appears in Dynamics CRM</td> <td valign="top" width="108">16:11:21</td> <td valign="top" width="363">3 Seconds from Start</td></tr> <tr> <td valign="top" width="350">Last Update Completes in Dynamics CRM</td> <td valign="top" width="108">16:11:22</td> <td valign="top" width="363">4 Seconds from Start</td></tr></tbody></table> <p>&nbsp;</p> <p><strong>Conclusion</strong></p> <p>If we are having updates every 5 seconds we would probably do away with the change detection and just schedule the updates every 5 seconds, thus reducing the load on the source. DS3 Server is also doing some clever stuff like stopping the trigger running another insert when detecting changes and making sure any updates that happened during a sync were processed.</p> <p>&nbsp;</p> <p>Real-Time Dynamics CRM synchronisation is possible and manageable, I set this up in about 15 minutes and ran the tests over an hour. When we implement&nbsp; these systems with our clients, we also run a reconciliation process which verifies all the updates and delete have been completed. This is a simple job we schedule nightly and allows us to also manage deletes elegantly and gives the business managers a solid reconciliation position.</p> <p>&nbsp;</p> <p>If you would like to discuss with us how you can increase the productivity of you Dynamics CRM solution through Real-Time integration, give us a call or email : <a href="mailto:support@simego.com">support@simego.com</a>.</p> Team Blog Fri, 01 Nov 2013 15:19:20 +0000 http://www.simego.com/Blog/2013/10/Ouvvi-Automation-Server-Instances info@simego.com (Sean Cleaver) Ouvvi Automation Server - Instances <p>Ouvvi Automation Server can now be licensed and configured to support multiple isolated environments on a single server. This is great if you want to separate your line of business applications or create a clear separation between DEV and PROD environments on the same machine.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_5c27ca56-a46a-401d-9bcd-db2527bd7897"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_1c8d2e6f-b88c-4a06-8c83-7b638ece86dc" alt="image" style="display: inline; border: 0px;" title="image" height="434" width="640" /></a></p> <p>&nbsp;</p> <p>Each Environment has an individual Database, Windows Service and Website therefore ensuring that each Instance/Tenant is completely independent of the others. This ensure that you can separate both the operation of the integration and who has access to the environment.</p> <p>&nbsp;</p> <p>Each Instance shares the Data Synchronisation Studio installation on the server.</p> <p>&nbsp;</p> <p>The number of instances on a physical machine is limited only by what is practical, since each instance requires it&rsquo;s own dedicated Windows Service&nbsp; the maximum we would recommend would be 20 instances on a single server.</p> <p>&nbsp;</p> <p>The instances are licensed individually if you would like to talk to us about this configuration please get in touch via our support desk <a href="mailto:support@simego.com">support@simego.com</a></p> Team Blog Fri, 18 Oct 2013 10:29:11 +0000 http://www.simego.com/Blog/2013/10/Synchronise-Active-Directory-Users-with-SharePoint-List info@simego.com (Sean Cleaver) Synchronise Active Directory Users with SharePoint List <p>Quick guide that shows how you can use Data Synchronisation Studio with your Active Directory and SharePoint to create a Staff Directory or similar from Data Stored within your Active Directory.</p> <p>&nbsp;</p> <p>First step is to create a “Contacts” List in the SharePoint Site you want to use to hold the data from Active Directory. Here we’re using SharePoint 2010 but you can use SharePoint 2007 and 2013.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_2795a281-1369-4158-adc7-838666fd8b51"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_e59669ba-424f-43dc-b5c6-e0f8a8b29149" width="644" height="196"></a> </p> <p>&nbsp;</p> <p>Next Step is to create a Column to store the Active Directory Identifier (DSID in Data Sync). Data Sync includes a Wizard that allows you to create a Read-only Column on your SharePoint list.</p> <p>&nbsp;</p> <p>Go to the Tools menu and Select “<strong>Create SharePoint Column</strong>” </p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_e5856256-1dd8-4a52-abcb-429eea75690b"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_00c3ea3d-a6fc-4809-9d4e-f48604d9a2b6" width="613" height="418"></a> </p> <p>&nbsp;</p> <p>This starts the Column Wizard.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d4c42261-a25d-44aa-8ed9-78c5c67577e3"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_7369c34c-71d8-4fb2-91b6-4d48ee3f03a0" width="510" height="393"></a> </p> <p>&nbsp;</p> <p>Enter your SharePoint Site URL and credentials (Leave Credentials blank for current logged on user).</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_735ea487-a513-4403-a0a0-e3f2693f25a9"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_22b54384-923b-4c6a-953d-3b7338aaa86d" width="510" height="393"></a> </p> <p>&nbsp;</p> <p>Choose the List that you created earlier.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_dac10e8c-b19e-416c-9233-83a4e68ca4d9"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_033cacd9-4cd1-4083-a45c-ff6f84e6bc1d" width="510" height="393"></a> </p> <p>&nbsp;</p> <p>Create the column as DSID and Type System.String, Hidden (DSID is a special column type known to Data Sync).</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_10f9964f-21ba-4368-88af-629dbe3fd69d"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_96a650f0-c84f-44c9-8eef-30f2c6e88c5b" width="510" height="393"></a> </p> <p>&nbsp;</p> <p>Click finish and Data Sync will add the DSID column to the SharePoint List.</p> <p>&nbsp;</p> <p>Next Step is to connect Data Source A in Data Sync to your Active Directory, Click “<strong>Connect Data Source</strong>” on Data Source A.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ac10729f-1b23-449f-a3cd-4116257e2c00"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_0148308d-a5e3-4550-9d9b-85d4b47e7dd3" width="298" height="516"></a> </p> <p>&nbsp;</p> <p>Then Locate the <strong>Active Directory Users</strong> Provider and enter the LDAP path to your Directory, this could be something like <strong>LDAP://dc=corp, dc=litware, dc=com</strong> or a server name or IP address like <strong>LDAP://yourdomaincontroller</strong> </p> <p>If You want to also filter the returned users to a specific Group you can also define the Group filter in the <strong>LDAPFilter</strong> property.</p> <p>&nbsp;</p> <p>For example this filter returns objects of Type User from OU <strong>office.simego.com</strong> where the users are members of the <strong>Simego Users</strong> group.</p> <p><strong>&amp;(objectClass=user)(memberOf=CN=Simego Users,CN=Users,DC=office,DC=simego,DC=com))</strong>&nbsp;</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_47211093-5923-4f67-9138-5da0bb4a59bc"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_8fc412eb-f7c6-480a-b2d1-e41cfa96feaa" width="644" height="356"></a></p> <p>&nbsp;</p> <p>Next Step connect Data Source B to your SharePoint List (Use the SharePoint (Client API) Provider with SharePoint 2010 and 2013)</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_3346ac9f-99eb-4bf6-b11d-555e4f313611"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_b08b6dd8-f7f3-40ca-ae5d-2e8c1350b348" width="644" height="356"></a> </p> <p>&nbsp;</p> <p>Locate the List you created earlier from the SharePoint Explorer.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f69bfdb7-28f0-4cad-9321-55515c9c4c74"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_125075b1-0620-4c48-b875-e7b87b8d8853" width="504" height="505"></a> </p> <p>&nbsp;</p> <p>Pressing OK, brings up Data Sync Automatic Mapping feature where Data Sync will automatically apply a pre-defied schema map. Select the <strong>AD to SharePoint Contact List (DSID)</strong> Map.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_45d4ed23-9b39-4c89-8923-ef588619a809"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_c89ed93c-7d5f-4b8e-9802-9b427cfa820a" width="382" height="312"></a> </p> <p>&nbsp;</p> <p>This then creates a pre-configured schema map ready to go.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d07c1a47-15f8-4aa3-9f66-31ec58f2d2b6"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_d447fe29-6a6f-45f7-9794-a5951d98b90f" width="644" height="396"></a> </p> <p>&nbsp;</p> <p>Now click <strong>Compare A-&gt;B</strong> to load the data from AD and SharePoint and work out the changes required to make both sides equal.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_7d3243d7-64d1-4a7b-884e-12276a3a06b9"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_fb1eea16-5d24-4616-8ae4-30b3819578f6" width="600" height="149"></a> </p> <p>&nbsp;</p> <p>Data Sync then display the results where you can view the data before it is coped to SharePoint.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_96108068-e640-4529-801a-5b3531d87eff"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_fed348ff-9c08-41fa-a22a-f9e0ce899f64" width="644" height="448"></a> </p> <p>&nbsp;</p> <p>Click Synchronise on the results window to open the synchronise data window.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_3102f1c6-952e-4769-95e5-aa8f801c4710"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_c38c1e75-5912-4977-97c6-10879e813eeb" width="401" height="112"></a> </p> <p>&nbsp;</p> <p>Click Start to actually Synchronise the Data with SharePoint.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_bcd8e660-d28f-4d8a-8a8f-46053b2e6972"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_1b9f8ef2-efea-49aa-a275-3a0b879ff7d5" width="446" height="322"></a> </p> <p>&nbsp;</p> <p>After a few Seconds the sync process will complete.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_de6f6f00-9545-44cb-921f-07294eb0d24e"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_dc97e612-2a5f-4bb6-a006-949c302248ab" width="446" height="322"></a> </p> <p>&nbsp;</p> <p>Now if you switch to SharePoint you will see your data created in your list.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_742b8996-941f-4aa5-917c-3a3b63b57d26"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_971a3cfc-8f64-4b2c-9d16-ce6b063928d6" width="644" height="295"></a> </p> <p>&nbsp;</p> <p>Now if you want to run this on a Schedule via Windows Task Scheduler you should stave the project and run the “<strong>Create Scheduled Task</strong>” option from the Tools Menu.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_47c002a9-a5aa-4339-ac52-f8ac02815c89"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_2a9df65c-977b-4095-a6d9-057db07c7493" width="540" height="419"></a> </p> <p>&nbsp;</p> <p>Select the schedule Click OK and enter your Password. Windows will then Run your project based on the Schedule you selected to keep your Data between AD and SharePoint up to date.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_5daafa3a-c6fd-4abd-89ac-161aae5fcd21"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_ce64f56c-568a-419b-83dd-6140b923ba54" width="482" height="338"></a></p> Team Blog Thu, 17 Oct 2013 21:02:34 +0000 http://www.simego.com/Blog/2013/10/Scheduled-Export-Import-for-Dynamics-CRM david@simego.com (David Evans) Scheduled Export/Import for Dynamics CRM <p>Often when working with Dynamics CRM there are other systems and processes that require data from Dynamics CRM. This is a task that can be easily completed by using DS3 and Ouvvi in three steps.</p> <p>&nbsp;</p> <p>For this example we wanted to export 3 items to Excel and one to XML at 6 o&rsquo;clock in the evening.</p> <p>1. All &lsquo;Open&rsquo; opportunities have not been modified in the last week.</p> <p>2. All new contacts (FirstName, LastName, email address and Telephone) that were created this month.</p> <p>3. All new accounts that were created this month (AccountNumber, Name and web address)</p> <p>&nbsp;</p> <p>For each of these exports it is necessary to complete 3 steps in DS3:</p> <p>1. Connect to the Dynamics CRM and the entity for export.</p> <p>2. Pick the columns for export and filter the rows being returned.</p> <p>3. Connect to the Excel or XML Files.</p> <p>&nbsp;</p> <p>We want to schedule these exports so the first thing is to create a Project in the Automation Server Ouvvi and then three empty DS3 Steps:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_8714e470-ae77-4816-9b57-c141d043daba"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_951894c5-e758-4575-b422-93668644d2bb" alt="image" style="display: inline; border: 0px;" title="image" height="283" width="680" /></a></p> <p>&nbsp;</p> <p>I open the first empty step &ldquo;Export Unmodified Opportunities&rdquo; in DS3 and connect to the Dynamics CRM 2013 Opportunity entity:</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_234987c8-79d5-4e8c-9f1d-653763d5f79b"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_f0e230ee-d2e1-4633-a4c4-6b89f6a838f0" alt="image" style="display: inline; border: 0px;" title="image" height="438" width="680" /></a></p> <p>&nbsp;</p> <p>I can then just drag and drop the columns onto the schema window until I am happy:</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ce000a89-da60-464e-9bde-c41fa665f939"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_14d321c0-7600-421b-bebe-fe2ae18f5373" alt="image" style="display: inline; border: 0px;" title="image" height="438" width="680" /></a></p> <p>&nbsp;</p> <p>Then I can use the &lsquo;Tools'-&gt;Create Excel Sheet&rsquo; to create the exact source schema in an excel file . DS3 will also automatically set the destination to that Excel sheet. <em>Interestingly, DS3 can create SQL Tables, SharePoint Lists, XML files etc from source schemas.</em></p> <p>&nbsp;</p> <p>We can then click, Compare A-&gt; B..Synchronise..Start and all of the Opportunities all be in the Excel sheet.</p> <p>&nbsp;</p> <p>In reality we want to filter the opportunities to just the ones in the last month. In Dynamics CRM we in an advanced find (or view a current view) we can export the filters by clicking the &lsquo;Export to XML&rsquo; to see the &lt;filter&gt; expression.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_816bff75-522b-48a8-be28-f7785dd75cb4"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_103faf94-169a-4aa4-9fae-3fcae5419408" alt="image" style="display: inline; border: 0px;" title="image" height="494" width="680" /></a></p> <p>&nbsp;</p> <p>We can copy the &lt;filter&gt; portion&nbsp;into the FilterXmlExpression property in the Dynamics CRM Source connection details in DS3:</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_e12f24af-ddf1-4b35-82ef-0267e4cd1ce3"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_4d4b5642-2f3e-48a3-bb76-8e53ed35ba5d" alt="image" style="display: inline; border: 0px;" title="image" height="333" width="607" /></a></p> <p>&nbsp;</p> <p>Previewing the Data again will show that we have filtered the opportunities down to this months opportunities:</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_9bf5a747-f35d-41c5-9f95-ec826c93fbd1"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_c1fad1dd-d56d-47c6-bf39-c272b5465d6e" alt="image" style="display: inline; border: 0px;" title="image" height="438" width="680" /></a></p> <p>&nbsp;</p> <p>Synchronising again will update the Excel file with the filtered rows. We can now just click the save button and the project will be saved inside Ouvvi and safly under version control.</p> <p>&nbsp;</p> <p>We can run the project from inside Ouvvi to make sure that the server connect to the Dynamics CRM instance and the target Excel file. We do this by Clicking on the &lsquo;Start Project&rsquo; button:</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_dc7bfb15-7a25-4271-84f7-9dad96c98d51"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_af3b4680-0c63-45ef-aae5-7086f436b406" alt="image" style="display: inline; border: 0px;" title="image" height="421" width="680" /></a></p> <p>&nbsp;</p> <p>We repeat this for the other exports.</p> <p>&nbsp;</p> <p>The next step is to schedule the project to run at 6 o&rsquo;clock every evening. Projects are started by triggers. To create a trigger click on the Trigger menu and then Add Trigger and make it a time trigger:</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_c4ebef1a-00f0-46c0-9c26-f5541e8a7f76"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_30028108-73cc-4ba9-af30-1e1b7b86730c" alt="image" style="display: inline; border: 0px;" title="image" height="469" width="680" /></a></p> <p>&nbsp;</p> <p>We can then go back to the &lsquo;Schedule Exports&rsquo; project and add the new trigger from the &lsquo;Triggers&rsquo; tab:</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_7c5c974f-bcac-4455-83d0-339022fc74ee"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_258d4e46-c108-4baa-9e42-b55fb881c257" alt="image" style="display: inline; border: 0px;" title="image" height="469" width="680" /></a></p> <p>&nbsp;</p> <p>The project will now run, exporting the three files every night (except Sat and Sun) at 18:00. We can use the &lsquo;Documentation&rsquo; feature to create some documentation:</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_7bc42452-2ed0-4706-9eb8-c321db0db4f4"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_8c774765-7c96-4489-92eb-74bfbafcaa5f" alt="image" style="display: inline; border: 0px;" title="image" height="811" width="680" /></a></p> Team Blog Thu, 17 Oct 2013 13:48:01 +0000 http://www.simego.com/Blog/2013/10/Capturx-Partners-with-Simego-for-More-Data-Integration-Options info@simego.com (Sean Cleaver) Capturx Partners with Simego for More Data Integration Options <p>Here at <a href="/Help/SharePoint/How-to-Export-Capturx-for-SharePoint-Data-to-SQL-Table-via-OData">Adapx</a> we work with 30+ partners to offer our customers easy Microsoft SharePoint integration into a range of systems like: SAP, IBM Maximo, EMC Documentum, Microsoft Dynamics, OSIsoft PI System, Oracle, Kronos, SalesForce, GeoPDF, and many more.</p> <p><br>Thanks to our new partner Simego, a UK-based independent software and custom-development services company, Capturx customers can now easily get their Microsoft SharePoint data exported and integrated into SQL table(s) via OData.</p> <p><br>As a quick refresher,&nbsp; Capturx for SharePoint enables teams to easily collect data with tablets or digital pens, and then send the data in real-time to their SharePoint Server. With Capturx, handwritten data is automatically captured, sent to SharePoint,&nbsp; converted to text, available for review, queued in automated workflow processes, and archived. From there, as mentioned above, integration into a range of systems is possible.</p> <p><br>With Simego's <a href="/Products/Data-Synchronisation-Studio">Data Synchronisation Studio</a>, Capturx customers can export data stored within their SharePoint&nbsp; Server and synchronize it with a SQL Server Database Table for reporting or archive purposes. The process is quick, easy, does not require special coding, and the Data Sync Project can be saved and scheduled to run at a specific time schedule either with Windows Task Scheduler, SQL Agent or Simego Automation Server (Ouvvi). For a step-by-step guide to see how you can go about configuring the integration <a href="/Help/SharePoint/How-to-Export-Capturx-for-SharePoint-Data-to-SQL-Table-via-OData">see here</a>. </p> <p>&nbsp; <p>If you'd like to learn more please contact a Capturx Solutions Engineer by emailing <a href="mailto:info@capturx.com">info@capturx.com</a>.</p> Team Blog Tue, 15 Oct 2013 20:29:31 +0000 http://www.simego.com/Blog/2013/10/Podio-Import-CSV-File info@simego.com (Sean Cleaver) Podio Import CSV File <p>This is an example of how you can use Data Synchronisation Studio to Import or Export a CSV File with Podio. You can use Data Sync to schedule this operation to happen at regular intervals to create an Integration between systems.</p> <p>&nbsp;</p> <p>First step is to connect Data Source A of Data Sync to your Source CSV File</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_bf20984b-23c7-4e0d-acef-3da7829c491f"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_d5e7e71e-0ce3-4f4d-be36-053c1f90563a" width="644" height="356"></a> </p> <p>&nbsp;</p> <p>This will then Open in Data Sync with a default schema</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_45964b30-2bc8-4911-9506-90e9c07c23fb"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_fc21627b-485d-4bf0-9f9b-bf68f8712499" width="644" height="302"></a> </p> <p>&nbsp;</p> <p>Click Preview A on the Schema Map to preview the Data and Check that it loads as expected.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_290603af-fac2-4486-8ed5-f36b8ac86f63"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_24d9221e-bb75-423c-9271-a16ad597e704" width="959" height="487"></a> </p> <p>&nbsp;</p> <p>Then in Podio create your App that will be used to store this information.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ce8f7069-16c0-4274-a566-1e4ad355b4ce"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_aed4fea5-dec3-4012-ab8b-ffd9cc988e23" width="639" height="484"></a> </p> <p>&nbsp;</p> <p>Next you need to connect Data Sync <strong>Data Source B</strong> to your <strong>Podio</strong> Account. First press the … button next to Credentials and sign-into your Podio Account. </p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_8ff2e388-8f1b-4461-aeb2-401296782604"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_68ff4281-da45-4c2d-9280-c05bed004a06" width="644" height="356"></a> </p> <p>&nbsp;</p> <p>Now Select your <strong>App</strong> from the App drop-down.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_9a0d9422-02e6-4bae-9011-95cf63aa0e86"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_093744f7-eada-46c8-856d-090596f87677" width="644" height="356"></a> </p> <p>&nbsp;</p> <p>Now define your schema map, some fields will map automatically others you may need to do manually by either drag and drop or selecting from the drop down.</p> <p>&nbsp;</p> <p>Important here you need to define a unique key that links the 2 systems together. Here we using the Podio <strong>external_id</strong> field to hold the <strong>ID</strong> value of our records from our source CSV file.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d33fddf3-9326-49b0-be2a-5a65755b8151"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_5259aaf8-f3cd-4503-b005-402b8dcbcb20" width="644" height="331"></a> </p> <p>&nbsp;</p> <p>Now click Compare A-&gt;B on the toolbar so that Data Sync reads the data and calculates the changes to apply to Podio. You will get all records here since the App is empty.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d9a747f0-add8-4a23-bd33-f855f533250c"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_f08b9048-bda6-4a1a-9597-017c2a25b1b8" width="604" height="484"></a> </p> <p>&nbsp;</p> <p>Click <strong>Synchronise</strong> and <strong>Start</strong> to copy the data over to Podio</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_7ef706c3-1b0f-4dfd-a2e4-b92fc4766363"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_2c521559-a338-4952-87a0-3f5b8bef64b6" width="446" height="322"></a> </p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_919379f7-135b-4e2b-a2ce-cf1ebde3df15"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_87a775d6-28b8-4e67-b90c-9d4357a5a8ac" width="446" height="322"></a> </p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_5630bae2-8a76-47dc-a88e-b393177edbd6"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_af449199-bb2e-4e35-a14f-f7708dde4c93" width="446" height="322"></a> </p> <p>&nbsp;</p> <p>After the process has finished you will see Podio Updated with your data.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d2a6e65c-cfa9-49a6-8310-a23bfa0a7f79"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_6e84b572-5419-4e69-b955-5d83cd21097d" width="644" height="482"></a> </p> <p>&nbsp;</p> <p>Now that this is setup and configured, it simple to save the project so that it can be run again and via the Data Sync Run Tool or Ouvvi Automation Server this can be configured to run at regular intervals.</p> Team Blog Tue, 15 Oct 2013 13:23:19 +0000 http://www.simego.com/Blog/2013/10/Dynamics-CRM-Online-2011-to-2013-API-Performance david@simego.com (David Evans) Dynamics CRM Online 2011 to 2013 API Import Performance <p>If you have upgraded your online instance to Dynamics CRM 2013 you may come across a reduction in performance when using the API to integrate external systems.</p> <p>&nbsp;</p> <p>Historically, Dynamics CRM Online (2011) has had the ability to &lsquo;throttle&rsquo; the amount of data being passed through the API but from our experience this has never been used. Dynamic CRM Online&nbsp; 2013 has throttling in place which can affect integration performance.</p> <p>&nbsp;</p> <p>To get a better idea of how an much of a performance difference this can make we are going to compare insert/update and delete performance of Dynamics CRM Online 2011 vs 2013</p> <p>&nbsp;</p> <h3>Conservative Performance</h3> <p>We created a file containing 50000 (49996) cheque records with minimal fields:</p> <p>Recipient<br />Amount<br />RequestId</p> <p>&nbsp;</p> <p>And we are going to insert it directly into a custom entity</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f937e245-5b5f-4dae-b58b-e5bdc9accda9"><img height="153" width="680" src="https://simegoimages.blob.core.windows.net/images/blog_12c6fbae-b13d-4ffa-bf8c-f74f3d809958" alt="image" border="0" style="display: inline; border-width: 0px;" title="image" /></a>&nbsp;</p> <p>&nbsp;</p> <p>50k Record Performance at 9:30 am EST with Simego DS3</p> <p>&nbsp;</p> <table border="0" width="674" cellpadding="2" cellspacing="0"> <tbody> <tr> <td width="301" valign="top">&nbsp;</td> <td width="130" valign="top"> <p align="center">Insert/s</p> </td> <td width="122" valign="top"> <p align="center">Update/s</p> </td> <td width="119" valign="top"> <p align="center">Delete/s</p> </td> </tr> <tr> <td width="301" valign="top"> <p align="center">Dynamics CRM Online (2011)</p> </td> <td width="130" valign="top"> <p align="center">506</p> </td> <td width="122" valign="top"> <p align="center">450</p> </td> <td width="119" valign="top"> <p align="center">446</p> </td> </tr> <tr> <td width="301" valign="top"> <p align="center">Dynamics CRM Online (2013)</p> </td> <td width="130" valign="top"> <p align="center">316</p> </td> <td width="122" valign="top"> <p align="center">354</p> </td> <td width="119" valign="top"> <p align="center">320</p> </td> </tr> <tr> <td width="301" valign="top"> <p align="center">Dynamics CRM 2013 On-Premise*</p> </td> <td width="130" valign="top"> <p align="center">912</p> </td> <td width="122" valign="top"> <p align="center">842</p> </td> <td width="119" valign="top"> <p align="center">694</p> </td> </tr> </tbody> </table> <p>&nbsp;</p> <p>*Run on DELL Desktop with SSD within a VM with 8Gb of memory</p> <p>&nbsp;</p> <h3>Deep Dive into Dynamics CRM Online 2013 Throttling</h3> <p>It has long been accepted that to achieve performance at the above 50/s level you will have to employ the following technologies:</p> <ol> <li>Parallel Loading </li> <li>Re-Use of Connection </li> <li>Re-Use of Authentication</li> </ol> <p>These three techniques have worked well against Dynamics CRM Online and 2011&hellip;&hellip;&hellip;until now.</p> <p>&nbsp;</p> <p>The Dynamics CRM 2013 SDK documentation states:</p> <p><em>Throttling of concurrent calls &ndash; for Microsoft Dynamics CRM Online there is a limit of 2 concurrent ExecuteMultipleRequest executions per organization. If that limit is exceeded, a &ldquo;Server Busy&rdquo; fault is thrown before the first request is ever executed. For an on-premises deployment, throttling is not enabled by default.</em>&nbsp;</p> <p>&nbsp;</p> <p>Deeper down the message stack this throws an error related to the SQL Governor.</p> <p>&nbsp;</p> <p><strong>Essentially, if your integration solution attempts to create a third concurrent connection: GAME OVER.</strong></p> <p><b>BUT this is configurable by Microsoft at an Organization level and you can request an amendment after discussions with the MS Online team:</b></p> <p><b><br /></b></p> <p><b>&nbsp;</b><a href="https://community.dynamics.com/crm/f/117/t/121248.aspx">https://community.dynamics.com/crm/f/117/t/121248.aspx</a></p> <p>&nbsp;</p> <p>If you dig deep enough and experiment long enough there is an interesting behaviour.</p> <p>&nbsp;</p> <p><strong>Throttling will not engage if the batch size is 10 or below. This means you can have as many parallel connections as you like without getting the &lsquo;Server Busy&rsquo; error.</strong></p> <p>&nbsp;</p> <p><strong>Here is the accompanying video demonstrating the difference in performance of many threads with small batches.</strong></p> <p>&nbsp;</p> <iframe height="676" width="900" name="wistia_embed" class="wistia_embed" scrolling="no" frameborder="0" allowtransparency="true" src="http://feeds.simego.com//fast.wistia.net/embed/iframe/z9n9zip6c0?videoFoam=true"></iframe> <script type="text/javascript"></script> <p>&nbsp;</p> <p>You can investigate these performance numbers yourself by trying <a target="_blank" href="http://www.siemgo.com">DS3 for free</a></p> Team Blog Mon, 14 Oct 2013 16:13:03 +0000 http://www.simego.com/Blog/2013/10/Import-Email-Messages-to-SharePoint-List info@simego.com (Sean Cleaver) Import Email Messages to SharePoint List <p>We have a great new provider in Data Sync that can be used to import email messages into a SharePoint List. You can even extract data from the message body and update SharePoint Metadata columns with the results.</p> <p>&nbsp;</p> <p>The Email provider is for Exchange Server only (sorry no generic IMAP support) and works with Office 365 as well as On Premise.</p> <p>&nbsp;</p> <p><em>Requires: Data Sync Release 3.0.752+ and Ouvvi Release: 2.0.264+</em></p> <p>&nbsp;</p> <p>This is an example of using Email with SharePoint to create a simple Purchase order Request/Approval type application. Users will Email the PO request and Data Sync will import them into SharePoint where a standard Approve/Reject workflow can be used to manage the approval process.</p> <p>&nbsp;</p> <p>Importing Email needs to be incremental so that we are not continuously downloading your mailbox each time. Therefore we’re going to use Ouvvi to manage the project. Ouvvi will automatically add some project properties to the project when the project is run which we can use to manage the point in time where we sync from.</p> <p>&nbsp;</p> <h4>First Create a new Ouvvi Project</h4> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_4174c014-2d8e-4f66-8aaf-9b191f1621f0"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_b5e4348d-9f67-49c3-abe6-2cea5e892382" width="554" height="484"></a> </p> <p>&nbsp;</p> <h4>Next add a Data Sync Step</h4> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_cafff4a7-f02a-418f-8db7-82651234113d"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_25f5b10c-8057-4b2e-b7bb-1dc484aba2c2" width="554" height="484"></a> </p> <p>&nbsp;</p> <h4>You can now open the Project from Ouvvi</h4> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_945fbd1f-9f70-4294-a7ac-787b3104431e"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_01491eb4-b4e5-4585-9c0a-5144f3af537c" width="554" height="484"></a> </p> <p>&nbsp;</p> <p>Now open this project in Data Sync and connect Data Source A to the Email Provider, enter your Exchange Server Credentials and URL to the Exchange Web Services.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_84c565b3-7581-4b54-ac44-01b294e22d94"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_479e7cec-73dd-4712-a465-ca78d43adf5a" width="644" height="342"></a> </p> <p>&nbsp;</p> <p>Once we have connected the Email provider, we need to adjust some settings. As we’re interested in PO requests we are going to filter the Mailbox and only return messages who’s Subject is ‘<strong>PO Request</strong>’.</p> <p>You will also notice a <strong>FilterByRecievedDateTime</strong> field we will look at that later.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ba3f5d3f-7201-4297-9ea8-9ed6ed819239"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_12dab1f3-f3b4-435e-935f-94da495adf84" width="342" height="559"></a> </p> <p>&nbsp;</p> <p>You can then Preview the Data to check that Data Sync returns the messages correctly.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_1b11caee-ceab-445f-83a7-c163d647b782"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_12d9da3e-a2f1-43b9-b956-c5976759b1b1" width="644" height="59"></a> </p> <p>&nbsp;</p> <p>Next we need to connect the Target to a SharePoint Document Library that were using to store the Email Messages. Use the SharePoint Client API provider since this has better support for SharePoint 2010 and 2013.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_c173f1d9-8206-4a4b-aaf4-22e6d8739faf"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_d889f124-54a5-45ea-b016-2e057bab0891" width="644" height="356"></a> </p> <p>&nbsp;</p> <p>Choose the List ‘<strong>PO Requests</strong>’ from SharePoint Object Explorer.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_45520abb-cb78-4ebc-bd7b-8f3401bd461a"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_ec61a594-6d1c-47e3-ab94-87b810c2515d" width="504" height="505"></a> </p> <p>&nbsp;</p> <p>We need a minimal Map for this to work however you can extend your SharePoint List with additional columns if you want to capture more information.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_6d2a5d7b-58bf-40af-b499-3dbd52f1f4e6"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_efad715d-1aad-4815-aae3-d540bddd4008" width="720" height="165"></a> </p> <p>&nbsp;</p> <p>Click <strong>Compare A-&gt;B</strong> to Load your Email and compare it with what’s in SharePoint.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_80bd6777-9597-4dfe-b664-e1416348962b"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_73f8a926-036a-4e8a-a998-9a6651ffdf94" width="519" height="228"></a> </p> <p>&nbsp;</p> <p>Then Synchronise the Results, In SharePoint you will see that the actual message is stored as a document which can be opened directly from SharePoint. From here you can setup a SharePoint Approval Workflow to approve or reject the PO Request.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_9d39e7ff-ba6b-4736-8053-bd2ed2eea3c9"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_7db1ca13-062b-48db-ae00-519ee034b905" width="644" height="172"></a> </p> <p>&nbsp;</p> <p>Next for Incremental Sync,&nbsp; Ouvvi will add the Following Properties to your Project when it’s opened/run and this is where we need a little magic. What we’re going to do is update the value of <strong>FilterByRecievedDateTime</strong> from the Value in the Project Properties.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_71d2bc89-def8-4821-b724-d86f8cb4de30"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_93ea2925-e7b7-48fb-ae74-87fba654d715" width="767" height="421"></a> </p> <p>&nbsp;</p> <p>Go to View-&gt;Project Automation Window to open the code editor for the automation steps.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_323b2225-8e46-47c2-8be7-dc105a4b9932"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_8e38e5aa-1ab9-40ff-b3c3-ff52e7c45ccb" width="354" height="176"></a> </p> <p>&nbsp;</p> <p>Click enable and enter 1 line of code to copy the value from the project properties into the DataSourceA <strong>FilterByRecievedDateTime</strong>&nbsp; property before the project runs.</p> <p>&nbsp;</p> <p>public override void Start()<br>{<br>&nbsp;&nbsp;&nbsp; DataSourceA.FilterByReceivedDateTime = Properties["Auto_LastSuccessfulRun"];<br>} <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_b22f1083-45e1-4e21-963e-5719d50ef49f"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_20cbff4b-a19b-4735-9fce-c034879febc0" width="644" height="470"></a> </p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p>&nbsp;</p> <p>Then finally set the <strong>SyncOption</strong> of the Project to <strong>SyncAtoBIncrmental</strong> to enable the Incremental support with the SharePoint Provider.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_93948123-0287-4ac1-bd55-bbcf5acf2b11"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_69fa8611-7468-41a3-96e1-54efe5346c18" width="407" height="484"></a> </p> <p>&nbsp;</p> <p>Now you can schedule this to run directly in Ouvvi by adding a Time based Trigger to the Project and watch as your PO Request emails are all fed nicely into your SharePoint List.</p> Team Blog Mon, 14 Oct 2013 16:02:25 +0000 http://www.simego.com/Blog/2013/10/Converting-CSV-file-to-XML info@simego.com (Sean Cleaver) Converting CSV file to XML <p>Quite often you need a quick and simple way to convert data in a CSV File format into a simple XML document. Typically you would probably write a small script or use a software application and get involved with schema’s etc.</p> <p><a href="http://www.simego.com/Products/Data-Synchronisation-Studio">Data Synchronisation Studio</a> makes the whole process quick and painless. </p> <p>&nbsp;</p> <p>Simply open Data Sync and Connect<strong> Data Source A</strong> to your source CSV File.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d4df0efd-c732-4500-858f-ddb87e356daf"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_27b7b776-3dc1-4d15-b03c-613e8ec03aba" width="644" height="357"></a> </p> <p>&nbsp;</p> <p>You can preview the Data to check that it is loading correctly.</p> <p>&nbsp;</p> <p></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_c6f23f72-8e81-4810-8215-8310a221c476"><img title="CropperCapture[2]" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="CropperCapture[2]" src="https://simegoimages.blob.core.windows.net/images/blog_807e3786-fd25-466e-ab58-4ce795c2b63f" width="559" height="484"></a> </p> <p>&nbsp;</p> <p>This Dataset does not contain a Unique ID key column so were going to use a Feature in Data Sync called Calculated Columns to decorate the data with new Data.</p> <p>&nbsp;</p> <p>Click the <strong>Fx</strong> button on the Data Source A Toolbar to add a new Function and enter the Name and Function like this.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_25cf7743-2cbe-4627-b1b5-44c237b92bd1"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_6991caf4-9a6b-427f-8e3b-17dedf6b03a9" width="456" height="331"></a> </p> <p>&nbsp;</p> <p><strong>NEXTINT()</strong> function returns a number for each row incrementing the value by 1 each time it is called.</p> <p>We can now adjust the schema Map how we want the target XML to be created, setting Data Types here will ensure the XML Schema contains the correct data type.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_daff1c96-95ce-4a8d-b89b-a49e07212a23"><img title="CropperCapture[3]" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="CropperCapture[3]" src="https://simegoimages.blob.core.windows.net/images/blog_87aa8b66-ea25-4e18-b8ae-8d61fb3e23bb" width="655" height="382"></a> </p> <p>&nbsp;</p> <p>Next to create the XML Target Data Source go to the Tools menu and select “Create XML Dataset” this will ask for a Filename and create an Empty XML Document.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_1ee7aeaa-2881-40ce-b7a2-d7fa37fbe045"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_7abf235d-540f-4b4c-9413-fe7422bee41f" width="500" height="488"></a> </p> <p>&nbsp;</p> <p>Press<strong> Compare A-&gt;B</strong> and Data Sync will work out the rows to <strong>ADD/UPDATE/DELETE</strong> from the XML document, then press <strong>Synchronise</strong> and <strong>Start</strong> to actually copy the data to the XML File.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_900b2e79-13e8-47bd-9827-5b1d5370ad46"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_98a4bc2e-1258-4e5c-856b-de32632faf28" width="644" height="434"></a> </p> <p>&nbsp;</p> <p>Once the Synchronisation process has completed your then left with an XML Document containing your data.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_3704f1a9-b0b7-41d4-addb-86e637ba4a4c"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_a1e63c02-baee-4aa7-bb9d-80434ffff52b" width="504" height="484"></a> </p> <p>&nbsp;</p> <p>Now that this is all setup and configured, you can easily save this as a project and either schedule it to run automatically or just run it ad-hoc when you need it.</p> Team Blog Mon, 14 Oct 2013 14:33:24 +0000 http://www.simego.com/Blog/2013/09/Data-Sync-Incremental-Sync-Support info@simego.com (Sean Cleaver) Data Sync - Incremental Sync Support <p>Data Sync&nbsp; (beta) now includes a new compare mode that allows you to easily implement incremental synchronisation scenarios. This makes it very easy to add/update a small-ish subset of records against a target system that may contain many millions of rows.</p> <p>&nbsp;</p> <p>For example if you have a 20k row set of inserts and updates to compare against a 200M data-store you can now use Data Sync and it will only compare the 20k items to workout the inserts and updates automatically. This makes it incredibly fast to apply a change to an existing large data-store.</p> <p>&nbsp;</p> <p>We have support for the following providers with the new Incremental Sync option :-</p> <p>&nbsp;</p> <p>MS SQL Server</p> <p>ODBC</p> <p>OleDb</p> <p>Podio</p> <p>SharePoint 2010-2013</p> <p>SharePoint Online / 365</p> <p>Dynamics CRM 2011 &ndash; 2013</p> <p>&nbsp;</p> <p>If you use this new feature with Data Sync Automation Server (Ouvvi) you can easily design an incremental project that only requests changes since a point in time from your data, compare this with the target and move the date along automatically. Allowing you to design effective real-time synchronisation of your systems with little effort.</p> <p>To find out more and discuss your requirements please contact us.</p> Team Blog Fri, 27 Sep 2013 14:45:54 +0000 http://www.simego.com/Blog/2013/09/Simego-DS3-Training-Days info@simego.com (Sean Cleaver) Simego DS3 Training Days <p>We’re announcing the availability of our new DS3 Training Days, If your a customer, consultant or partner why not register your interest to attend one of our Free to Attend Group Training Days with our Team. </p> <p>&nbsp;</p> <p>On the Day you will learn more about DS3 and Automation Server, tips and tricks and get a chance to discuss your integration questions directly with the team.</p> <p>&nbsp;</p> <p>The Training Days are hosted at our Offices in East Sussex, UK and are free to attend.</p> <p>&nbsp;</p> <p>To register for a training day please email us at <a href="mailto:info@simego.com">info@simego.com</a> or call the office 01424 863450 for more information.</p> Team Blog Wed, 18 Sep 2013 12:27:49 +0000 http://www.simego.com/Blog/2013/09/Partner-C5-Alliance-Group-Leverages-Data-Sync-and-Ouvvi-as-a-IBM-DataMirror-Replacement info@simego.com (Sean Cleaver) Partner C5 Alliance Group Leverages Data Sync and Ouvvi as a IBM DataMirror Replacement <p>Our Partner <a href="http://c5.je/SuccessStories/Pages/Index.aspx">C5 Alliance Group</a> in the Channel Islands selects Data Sync and Ouvvi as an effective replacement for an aging IBM DataMirror installation.&nbsp; The solution keeps an MS SQL Database up to date from an iSeries Line of Business Application.</p> Team Blog Fri, 13 Sep 2013 15:53:00 +0000 http://www.simego.com/Blog/2013/09/Ouvvi-Program-Trigger info@simego.com (Sean Cleaver) Ouvvi :: Program Trigger <p>We&rsquo;ve added a new Trigger to Ouvvi that calls an external program to evaluate whether the trigger should be fired. This enables you to create a simple script like program that simply returns a Date Time value of the last change to fire the Trigger.</p> <p>&nbsp;</p> <p>A very simple .NET Console Application is all that&rsquo;s required, you simply write the code to discover the changes and return the Time of the Last Change to the Console Output.</p> <p>&nbsp;</p> <pre class="csharpcode"><span class="kwrd">using</span> System; <span class="kwrd">namespace</span> ExternalTrigger { <span class="kwrd">class</span> Program { <span class="kwrd">static</span> <span class="kwrd">void</span> Main(<span class="kwrd">string</span>[] args) { Console.WriteLine(DateTime.Now.ToString(<span class="str">"yyyy-MM-dd HH:mm:ss"</span>)); } } } </pre> <style type="text/css"><!-- .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } --></style> <p>&nbsp;</p> <p>Then you create the <strong>Program Trigger</strong> in Ouvvi, set the Program to Execute and attach the projects you want to execute when this trigger changes to this new Trigger.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_79345539-6ec9-45ce-9bde-43256f780538"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_8f305913-d3aa-4f3c-b3b5-6db42b2325a5" alt="image" style="display: inline; border: 0px;" title="image" height="406" width="644" /></a></p> <p>&nbsp;</p> <p>The Program should be fast, if the Trigger Evaluation time is long it will hold up the Trigger Thread in Ouvvi and impact the evaluation of the other triggers so it is advisable that this is a very short operation.</p> Team Blog Fri, 13 Sep 2013 13:01:25 +0000 http://www.simego.com/Blog/2013/09/Start-Ouvvi-Project-Programmatically info@simego.com (Sean Cleaver) Start Ouvvi Project Programmatically <p>Projects you have loaded and configured in Ouvvi Automation Server can be started and monitored from external processes on any machine that has network connectivity.</p> <p>For example this project here can be started programmatically with a very simple script or program.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_b8d3c683-4e21-494e-bc83-e487ea6d1e7f"><img title="image" style="display: inline; border: 0px;" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_fd81d8a9-b10f-4dca-9028-ee711814b79d" width="644" height="327" /></a></p> <p>&nbsp;</p> <p>To start an Ouvvi Project programmatically is as simple as issuing a HTTP Request to the Start Endpoint i.e.</p> <p>This will Start Project &lsquo;1&rsquo; on our DEV instance of Ouvvi.</p> <p><span style="text-decoration: underline;">http://dev.corp.simego.com/Project/Start/1</span></p> <p>&nbsp;</p> <p>This returns a simple XML Document with the result and Instance ID of the project.</p> <pre class="csharpcode"><span class="kwrd">&lt;</span><span class="html">result</span> <span class="attr">ProjectID</span><span class="kwrd">="1"</span> <span class="attr">InstanceID</span><span class="kwrd">="11"</span> <span class="attr">Status</span><span class="kwrd">="OK"</span><span class="kwrd">/&gt;</span></pre> <p>Then to Query the result of this project instance you would call the Project Query Endpoint passing the Instance ID returned before.</p> <p><span style="text-decoration: underline;">http://dev.corp.simego.com/Project/Query/11</span></p> <p>&nbsp;</p> <p>This then returns a simple XML Document with the Result and Log Details.</p> <pre class="csharpcode"><span class="kwrd">&lt;</span><span class="html">Results</span> <span class="attr">ID</span><span class="kwrd">="11"</span> <span class="attr">ProjectID</span><span class="kwrd">="1"</span> <span class="attr">StatusID</span><span class="kwrd">="1"</span> <span class="attr">Status</span><span class="kwrd">="Success"</span> <span class="attr">StartTime</span><span class="kwrd">="2013-09-09 14:41:41.053"</span> <span class="attr">EndTime</span><span class="kwrd">="2013-09-09 14:42:00.103"</span><span class="kwrd">&gt;</span> <span class="kwrd">&lt;</span><span class="html">Result</span> <span class="attr">ID</span><span class="kwrd">="21"</span> <span class="attr">StepID</span><span class="kwrd">="1"</span> <span class="attr">StepName</span><span class="kwrd">="ProductsToProducts"</span> <span class="attr">StatusID</span><span class="kwrd">="0"</span> <span class="attr">Status</span><span class="kwrd">="Idle"</span> <span class="attr">StartTime</span><span class="kwrd">="2013-09-09 14:41:41.087"</span> <span class="attr">EndTime</span><span class="kwrd">="2013-09-09 14:41:42.350"</span><span class="kwrd">/&gt;</span> <span class="kwrd">&lt;/</span><span class="html">Results</span><span class="kwrd">&gt;</span></pre> <p>&nbsp;</p> <p>This makes in incredibly easy to integrate Ouvvi Project execution from your client applications. For example this could be used to start SSIS Projects on the SQL Server Integration Server by calling a simple HTTP Request.</p> <p>&nbsp;</p> <p>For example below is sample code for a Console Application that starts an Ouvvi project and waits for it to complete.</p> <p>&nbsp;</p> <pre class="csharpcode"><span class="kwrd">using</span> System; <span class="kwrd">using</span> System.Net; <span class="kwrd">using</span> System.Threading; <span class="kwrd">using</span> System.Xml; <span class="kwrd">namespace</span> ConsoleApplication1 { <span class="kwrd">class</span> Program { <span class="kwrd">static</span> <span class="kwrd">int</span> Main(<span class="kwrd">string</span>[] args) { <span class="kwrd">int</span> projectID = 1; <span class="rem">//Start the Project</span> <span class="rem">// http://dev.corp.simego.com/Project/Start/1</span> <span class="rem">//</span> <span class="rem">//returns &lt;result ProjectID="1" InstanceID="11" Status="OK"/&gt; </span> XmlDocument document = <span class="kwrd">new</span> XmlDocument(); document.XmlResolver = <span class="kwrd">new</span> XmlUrlResolver { Credentials = CredentialCache.DefaultCredentials }; document.Load(<span class="kwrd">string</span>.Format(<span class="str">"http://dev.corp.simego.com/Project/Start/{0}"</span>, projectID)); <span class="kwrd">long</span> instanceID = -1; <span class="kwrd">string</span> result = <span class="str">"NOT_OK"</span>; var resultNode = document.SelectSingleNode(<span class="str">"/result/@Status"</span>); <span class="kwrd">if</span> (resultNode != <span class="kwrd">null</span>) result = resultNode.InnerText; var instanceNode = document.SelectSingleNode(<span class="str">"/result/@InstanceID"</span>); <span class="kwrd">if</span> (instanceNode != <span class="kwrd">null</span>) instanceID = Convert.ToInt64(instanceNode.InnerText); Console.WriteLine(<span class="str">"Project: {0} Started: {1}, Instance ID: {2}"</span>, projectID, result, instanceID); <span class="rem">//Query the Project and Wait for Completion</span> <span class="rem">//http://dev.corp.simego.com/Project/Query/10</span> <span class="rem">//returns</span> <span class="rem">//&lt;Results ID="14" ProjectID="1" StatusID="1" Status="Success" StartTime="2013-09-09 15:04:03.827" EndTime="2013-09-09 15:04:05.213"&gt;</span> <span class="rem">//&lt;Result ID="24" StepID="1" StepName="ProductsToProducts" StatusID="0" Status="Idle" StartTime="2013-09-09 15:04:03.903" EndTime="2013-09-09 15:04:05.213"/&gt;</span> <span class="rem">//&lt;/Results&gt;</span> <span class="kwrd">while</span> (<span class="kwrd">true</span>) { <span class="kwrd">string</span> projectResult = <span class="kwrd">null</span>; XmlDocument statusdocument = <span class="kwrd">new</span> XmlDocument(); statusdocument.XmlResolver = <span class="kwrd">new</span> XmlUrlResolver { Credentials = CredentialCache.DefaultCredentials }; statusdocument.Load(<span class="kwrd">string</span>.Format(<span class="str">"http://dev.corp.simego.com/Project/Query/{0}"</span>, instanceID)); var projectResultNode = statusdocument.SelectSingleNode(<span class="str">"/Results/@Status"</span>); <span class="kwrd">if</span> (projectResultNode != <span class="kwrd">null</span>) projectResult = projectResultNode.InnerText; <span class="kwrd">if</span> (!<span class="kwrd">string</span>.IsNullOrEmpty(projectResult)) { <span class="kwrd">if</span> (projectResult.Equals(<span class="str">"Success"</span>)) { Console.WriteLine(<span class="str">"Project Completed Succesfully."</span>); <span class="kwrd">return</span> 0; } <span class="kwrd">if</span> (projectResult.Equals(<span class="str">"Failure"</span>)) { Console.WriteLine(<span class="str">"Project Failed."</span>); <span class="kwrd">return</span> 1; } Console.WriteLine(<span class="str">"Project Status:{0}"</span>, projectResult); } Thread.Sleep(250); } } } } </pre> <style type="text/css"><!-- .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } --></style> <p>&nbsp;</p> <p> <style type="text/css"><!-- .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } --></style> </p> Team Blog Mon, 09 Sep 2013 15:19:31 +0000 http://www.simego.com/Blog/2013/09/If-only-Ouvvi-could-Execute-my-SSIS-Packages info@simego.com (Sean Cleaver) If only Ouvvi could Execute my SSIS Packages <p>New feature coming to Ouvvi Automation Server, the ability to execute SSIS Packages. This is great because now it means you can easily monitor, track changes and start SSIS packages right from the Ouvvi Automation Server Console.</p> <p>&nbsp;</p> <p>It gets better, Ouvvi can also be configured to execute tasks on different machines/agents so you can install an Agent on the SQL Server to execute SSIS packages and set an Execution Condition for the SSIS Step on the SQL Server. (Think about that … Ouvvi can run tasks on any machine you have an Ouvvi Service Agent so you can distribute tasks across the enterprise and join them up!)</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_3209f8b9-9add-4682-8332-220f1369c555"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_884a5ef8-4ae2-4109-95b2-7ea1ff946a72" width="644" height="306"></a></p> Team Blog Fri, 06 Sep 2013 11:25:17 +0000 http://www.simego.com/Blog/2013/09/Charts-and-Office-365-SharePoint-Online info@simego.com (Sean Cleaver) Charts and Office 365 SharePoint Online <p>So you have an Office 365 account that comes with SharePoint 365 and want to do something useful with it? Why not create a Dashboard and Chart all your important Business Metrics for your users to see in real-time.</p> <p>Ah! But where are the Charts?</p> <p>I couldn’t find any decent Chart capability in SharePoint 365 so thought how could we easily get some charting going.</p> <p>&nbsp;</p> <p>Over to the great JavaScript Charting Library <a href="http://www.chartjs.org/">Chart.js</a>&nbsp; by <a href="http://www.nickdownie.com/">Nick Downie</a>. Embedding this Chart Library in your site and a little JavaScript code it’s possible to create a Dynamic Chart that renders from the Data stored in SharePoint lists within your SharePoint Site.</p> <p>&nbsp;</p> <p><strong>Example Chart displayed from the List “SalesData”.</strong></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_c3767e16-0613-448b-a33d-2b5489203e26"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_cf3bcb8d-27cf-4ff3-a03a-b69ff86077be" width="644" height="453"></a> </p> <p>&nbsp;</p> <p>So how does this work? These are the Steps you need to follow to create a Chart similar to the screenshot above.</p> <p>&nbsp;</p> <ol> <li>Upload the <strong>Chart.js</strong> to a Document Library <li>Create a SharePoint <strong>List</strong> that contains the Data you want to Chart. <li>Create a SharePoint <strong>Page</strong> to stick the Chart on. <li>Add a <strong>Script Editor</strong> control to the Page <li>Add the <strong>JavaScript code</strong> the create the Chart</li></ol> <p>&nbsp;</p> <h4>Step 1</h4> <p>Create a new Document Library called ‘Assets’ in your Site to hold the Chart.js.go to <a href="http://www.chartjs.org/">http://www.chartjs.org/</a> and download the Chart code and upload the Chart.js to this document Library.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_5446b278-d138-446e-ae87-e821953c07c7"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_c7f5fa3a-5cfd-423e-ab69-2aafb2f5af67" width="1028" height="583"></a> </p> <p>&nbsp;</p> <h4>Step 2</h4> <p>Create a List that holds the Chart Data, this example simply has a List that maps total units against months to simulate sales volume over a period.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_b15a2bf4-8df1-4ceb-8343-0e37d4a2a62d"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_021f11e2-dcc8-4ff9-ad65-42c1cdf137f6" width="442" height="484"></a> </p> <p>&nbsp;</p> <h4>Step 3</h4> <p>Create a new SharePoint Page to Host the Chart, select Add Page from the Admin Menu and create the Page.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f4f85878-4152-46e5-ab7e-0dfb80c0e75c"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_3ec08a14-3838-4718-a26c-9e7751053b9f" width="148" height="244"></a> </p> <p>&nbsp;</p> <p></p> <h4>Step 4</h4> <p>Edit the Page Design and Add a Script Editor Web Part to the Page.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d61716a1-f805-4db6-8e07-e210533a41fc"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_503d014c-d904-4cbb-a635-9ced085ab28a" width="644" height="233"></a> </p> <p>&nbsp;</p> <h4>Step 5</h4> <p>Add the JavaScript Code below to the Script Editor to read the List Content and render the Chart.</p> <p>This Code uses a <strong>Canvas</strong> element to draw the chart on the page, queries the List Data via the SharePoint Client Object Model and passes the data to the Chart.js.</p> <p>&nbsp;</p> <p>You will likely need to modify the settings at the start of the code for where the Chart.js is and other details about your List but other than that it should just work.</p> <p>&nbsp;</p><pre class="csharpcode"> <span class="kwrd">&lt;</span><span class="html">canvas</span> <span class="attr">id</span><span class="kwrd">="myChart"</span> <span class="attr">width</span><span class="kwrd">="600"</span> <span class="attr">height</span><span class="kwrd">="400"</span><span class="kwrd">&gt;&lt;/</span><span class="html">canvas</span><span class="kwrd">&gt;</span> <span class="kwrd">&lt;</span><span class="html">script</span> <span class="attr">type</span><span class="kwrd">="text/javascript"</span><span class="kwrd">&gt;</span> <span class="kwrd">var</span> collListItem; <span class="rem">//ListItems</span> <span class="kwrd">var</span> chartX = []; <span class="rem">//X-Axis Labels</span> <span class="kwrd">var</span> chartY = []; <span class="rem">//Y-Axis Values</span> <span class="kwrd">var</span> chartJs = <span class="str">"/sales/assets/chart.js"</span>; <span class="kwrd">var</span> listName = <span class="str">"SalesData"</span>; <span class="rem">//Data List Name</span> <span class="kwrd">var</span> xAxisName = <span class="str">"Title"</span>; <span class="rem">//X-Axis Label Names from List</span> <span class="kwrd">var</span> yAxisName = <span class="str">"Value"</span>; <span class="rem">//Y-Axis Values from List</span> <span class="kwrd">var</span> chartId = <span class="str">"myChart"</span>; <span class="rem">//Chart Canvas Div</span> SP.SOD.executeOrDelayUntilScriptLoaded(<span class="kwrd">function</span> () { <span class="kwrd">var</span> clientContext = <span class="kwrd">new</span> SP.ClientContext.get_current(); <span class="kwrd">var</span> oList = clientContext.get_web().get_lists().getByTitle(listName); <span class="kwrd">var</span> camlQuery = <span class="kwrd">new</span> SP.CamlQuery(); camlQuery.set_viewXml(<span class="str">'&lt;View&gt;&lt;/View&gt;'</span>); <span class="kwrd">this</span>.collListItem = oList.getItems(camlQuery); clientContext.load(collListItem); clientContext.executeQueryAsync(<span class="kwrd">function</span> () { <span class="kwrd">var</span> listItemEnumerator = collListItem.getEnumerator(); <span class="rem">//Create Points from ListData</span> <span class="kwrd">while</span> (listItemEnumerator.moveNext()) { <span class="kwrd">var</span> oListItem = listItemEnumerator.get_current(); chartX.push(oListItem.get_item(xAxisName)); chartY.push(oListItem.get_item(yAxisName)); } <span class="rem">//Load Chart JS</span> loadJS(chartJs, <span class="kwrd">function</span> () { <span class="rem">//Generate Data</span> <span class="kwrd">var</span> data = { labels: <span class="kwrd">this</span>.chartX, datasets: [ { data: <span class="kwrd">this</span>.chartY } ] }; <span class="rem">//Display Chart</span> <span class="kwrd">var</span> ctx = document.getElementById(chartId).getContext(<span class="str">"2d"</span>); <span class="kwrd">var</span> myNewChart = <span class="kwrd">new</span> Chart(ctx).Bar(data); }); }, <span class="kwrd">null</span>); }, <span class="str">'SP.js'</span>); <span class="kwrd">function</span> loadJS(src, callback) { <span class="kwrd">var</span> s = document.createElement(<span class="str">'script'</span>); s.src = src; s.async = <span class="kwrd">true</span>; s.onreadystatechange = s.onload = <span class="kwrd">function</span> () { <span class="kwrd">var</span> state = s.readyState; <span class="kwrd">if</span> (!callback.done &amp;&amp; (!state || /loaded|complete/.test(state))) { callback.done = <span class="kwrd">true</span>; callback(); } }; document.getElementsByTagName(<span class="str">'head'</span>)[0].appendChild(s); } <span class="kwrd">&lt;/</span><span class="html">script</span><span class="kwrd">&gt;</span> </pre> <style type="text/css">.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } </style> <p>&nbsp;</p> <p>There you have it a way to Chart data stored within your SharePoint Site all without purchasing or installing any sever side components.</p> Team Blog Tue, 03 Sep 2013 13:16:43 +0000 http://www.simego.com/Blog/2013/09/Using-Data-Sync-to-Screen-Scrape-Data-from-Websites info@simego.com (Sean Cleaver) Using Data Sync to Screen Scrape Data from Websites <p>Data Synchronisation Studio can be a very useful tool to extract structured data from a typically unstructured source such as the huge resource of the Internet.</p> <p>In this example we will show you how in about 10 minutes you can create a custom data sync data source that lists the Free Email Domain names from a List on this website <a href="http://www.joewein.de/sw/spam-freemailer.htm">http://www.joewein.de/sw/spam-freemailer.htm</a></p> <p>&nbsp;</p> <h4>Step 1 </h4> <p>Define the Source Schema Using the Empty Data source Provider.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_1494cf67-6b04-4e67-8cea-cde1037385d6"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_1362045a-14d1-40f1-af5d-3d21547bd615" width="914" height="505"></a> </p> <p>Create a single column in the Columns collection called “DomainName” this will return the Domain Name from the Website.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_313402e8-b670-48a6-96cc-45a620e1590a"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_1c32c78e-5330-44a1-bbc9-0e9f6fdba53b" width="511" height="377"></a> </p> <p>&nbsp;</p> <p>This results a very simple 1 column schema and currently returns no results.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_406a9ffc-e3d2-4622-acc7-28516bbaf84b"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_c4a93e84-564c-4abf-a997-e38128639040" width="644" height="185"></a> </p> <p>&nbsp;</p> <h4>Step 2 – Write the Code</h4> <p>Next we need to write the code that will override the row loading in Data Sync, download the webpage and extract all the domain names from the page.</p> <p>&nbsp;</p> <p>Lucky for us this is actually a very trivial process.</p> <p>&nbsp;</p> <ol> <li>Create a List&lt;string&gt; to contain all the Domain Names</li> <li>In the Setup() Method load and parse the Data.</li> <li>In the GetDataTable() method return the row data from the List&lt;string&gt; in Step 1.</li></ol> <p>&nbsp;</p> <p>The parse code below simply returns each line as a new domain name between the &lt;pre&gt;&lt;/pre&gt; tags in the web page.</p> <p>&nbsp;</p> <p>The GetDataTable() code is mainly boiler plate code the main part of interest is in the select statement where we match the requested “DomainName” column.</p> <p>&nbsp;</p><pre class="csharpcode"><span class="preproc">#region</span> Usings <span class="kwrd">using</span> System; <span class="kwrd">using</span> System.Collections.Generic; <span class="kwrd">using</span> System.Data; <span class="kwrd">using</span> System.Globalization; <span class="kwrd">using</span> System.Text; <span class="kwrd">using</span> System.Linq; <span class="kwrd">using</span> Simego.DataSync; <span class="preproc">#endregion</span> <span class="kwrd">using</span> Simego.DataSync.Providers.Null; <span class="rem">/// &lt;summary&gt;</span> <span class="rem">/// This module of code allows you to define some columns in code or Dynamic Columns</span> <span class="rem">/// To Create new Dynamic Columns simply expose them as properties in this class.</span> <span class="rem">/// &lt;/summary&gt;</span> <span class="kwrd">partial</span> <span class="kwrd">class</span> DataSourceRowOverride : Simego.DataSync.DynamicColumns.DataSourceRowInternal <span class="rem">//Do Not Change This Line</span> { <span class="kwrd">private</span> List&lt;<span class="kwrd">string</span>&gt; domainList = <span class="kwrd">new</span> List&lt;<span class="kwrd">string</span>&gt;(); <span class="kwrd">public</span> <span class="kwrd">override</span> DataTableStore GetDataTable(DataTableStore dt, IDataSourceReader reader) { <span class="rem">//Get the Actual Reader Instance.</span> NullDataSourceReader SourceReader = (NullDataSourceReader)reader; <span class="rem">//Create a Schema Mapping Helper</span> DataSchemaMapping mapping = <span class="kwrd">new</span> DataSchemaMapping(reader.SchemaMap, reader.Side); <span class="kwrd">for</span>(<span class="kwrd">int</span> i=0; i&lt;domainList.Count; i++) { var row = dt.NewRow(); <span class="kwrd">foreach</span> (DataSchemaItem item <span class="kwrd">in</span> reader.SchemaMap.GetIncludedColumns()) { <span class="kwrd">string</span> columnName = mapping.MapColumnToDestination(item); <span class="kwrd">switch</span>(columnName) { <span class="kwrd">case</span> <span class="str">"ID"</span>: { row[item.ColumnName] = DataSchemaTypeConverter.ConvertTo(i, item.DataType); <span class="kwrd">break</span>; } <span class="kwrd">case</span> <span class="str">"Domain"</span>: { row[item.ColumnName] = DataSchemaTypeConverter.ConvertTo(domainList[i], item.DataType); <span class="kwrd">break</span>; } <span class="kwrd">default</span>: { <span class="kwrd">break</span>; } } } <span class="kwrd">if</span> (dt.Rows.Add(row) == DataTableStore.ABORT) <span class="kwrd">break</span>; } <span class="kwrd">return</span> dt; } <span class="kwrd">public</span> <span class="kwrd">override</span> <span class="kwrd">void</span> Setup() { <span class="rem">//Download the WebPage</span> System.Net.WebClient webClient = <span class="kwrd">new</span> System.Net.WebClient(); <span class="kwrd">string</span> domainListContent = webClient.DownloadString(<span class="str">"http://www.joewein.de/sw/spam-freemailer.htm"</span>); <span class="kwrd">if</span> ( !<span class="kwrd">string</span>.IsNullOrEmpty(domainListContent)) { <span class="rem">//Parse the WebPage for Data</span> <span class="kwrd">string</span> line = <span class="kwrd">string</span>.Empty; <span class="kwrd">bool</span> start = <span class="kwrd">false</span>; <span class="kwrd">using</span>(var reader = <span class="kwrd">new</span> System.IO.StringReader(domainListContent)) { <span class="kwrd">while</span>((line = reader.ReadLine()) != <span class="kwrd">null</span>) { <span class="kwrd">if</span> ( !<span class="kwrd">string</span>.IsNullOrWhiteSpace(line)) { <span class="kwrd">if</span> ( line.Contains(<span class="str">"&lt;pre&gt;"</span>) || line.Contains(<span class="str">"&lt;/pre&gt;"</span>)) { start = !start; <span class="kwrd">continue</span>; } <span class="kwrd">if</span> ( start ) { domainList.Add(line.Trim()); } } } } } } } </pre> <style type="text/css">.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } </style> <p>&nbsp;</p> <p>Then previewing the data you see a nice up to date list created dynamically from the web page.</p> <p>&nbsp;</p> <p></p> <p></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_dd01b84d-aa82-4d7d-9779-ee3eecd7277d"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_2c289fe7-5c12-4204-8854-884cadaf8e76" width="413" height="736"></a> </p> <p>&nbsp;</p> <p>As you can see from a little bit of upfront coding 1 hour maximum you can create an integration of systems that would otherwise take take somebody probably over 1 hour everyday to update.</p> Team Blog Mon, 02 Sep 2013 15:29:37 +0000 http://www.simego.com/Blog/2013/08/System-Documentation-with-Ouvvi-Automation-Server info@simego.com (Sean Cleaver) System Documentation with Ouvvi <p>Do you have up to date and accurate integration documentation for your systems? If your integration is like most businesses the details are all hidden is some developer/consultant head. This is a risk to your business your systems integration becomes some magic black box and knowledge becomes lost over time.</p> <p>&nbsp;</p> <ol> <li>How can you manage change?</li> <li>What happens 6 months from now when you need to re-visit the integration?</li> <li>Has the Developer left the company?</li> </ol> <p>Now with Ouvvi Automation Server you can automatically generate compressive Documentation of your system integration detailing what connects to what and how everything is mapped. Create a valuable resource for your business.</p> <p>&nbsp;</p> <p>This documentation can then be either saved in your favourite Document Management System (SharePoint?) or further edited in Word to match your corporate requirements.</p> <p>&nbsp;</p> <p>Generating the Documentation is as Simple as clicking &ldquo;Documentation&rdquo; on the Ouvvi Home page.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_69a3a3fc-3fb2-4a1c-bb38-7b54dc8222d4"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_598560a8-0dd2-4991-9760-e69c50d314f0" alt="image" style="display: inline; border: 0px;" title="image" height="85" width="644" /></a></p> <p>&nbsp;</p> <p>This then creates a HTML Document that can be opened with Microsoft Word and saved as a PDF etc.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_10a4c519-3e41-4e63-bd8d-4f01abb23210"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_c2dd1faa-5d34-458b-b030-e0a16d56e53e" alt="image" style="display: inline; border: 0px;" title="image" height="484" width="551" /></a></p> <p>&nbsp;</p> <p>Sample Documentation opened in Word.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_70010270-ce5a-4821-a265-c51b3f8d112b"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_fcec8724-00e8-4514-83cb-2bafbe6fe7db" alt="image" style="display: inline; border: 0px;" title="image" height="484" width="566" /></a></p> Team Blog Thu, 29 Aug 2013 15:11:35 +0000 http://www.simego.com/Blog/2013/08/Noventri-Partners-with-Simego-for-Digital-Signage-Data-Integration info@simego.com (Sean Cleaver) Noventri - Partners with Simego for Digital Signage Data Integration <p>Digital Signage Company <a href="http://www.noventri.com/">Noventri</a> partners with Simego to leverage Data Synchronisation Studio to integrate 3rd party systems (SharePoint, Exchange, XML etc) with their product.</p> <p><a href="http://www.noventri.com/" title="http://www.noventri.com/">http://www.noventri.com/</a></p> Team Blog Thu, 29 Aug 2013 14:59:27 +0000 http://www.simego.com/Blog/2013/08/C5-Alliance-Group-Partners-with-Simego-for-Reliable-and-Trusted-Data-Integration info@simego.com (Sean Cleaver) C5 Alliance Group - Partners with Simego for Reliable and Trusted Data Integration <p><a href="http://c5.je/">C5 Alliance Group</a> partners with Simego to Leverage Data Synchronisation Studio and Ouvvi Automation Server with Client Projects.</p> <p><a href="http://c5.je/SuccessStories/Pages/Index.aspx">http://c5.je/SuccessStories/Pages/Index.aspx</a></p> Team Blog Thu, 29 Aug 2013 14:54:24 +0000 http://www.simego.com/Blog/2013/08/Version-History-with-Ouvvi-Automation-Server info@simego.com (Sean Cleaver) Version History with Ouvvi Automation Server <p>We added a new feature to Ouvvi to support Audit and Version History requirements, now when a Project Step is edited the system automatically saves the old version in the step history. </p> <p>&nbsp;</p> <p>This is an important feature in any company where it is vital you can track changes to your production systems, ensure people are accountable and have the ability to revert changes that were not intended quickly and easily.</p> <p>&nbsp;</p> <p>For Example if we have a Data Sync Step like this we can simply press the History menu button to access all the changes made to this step.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_b0667e93-4663-477b-93b6-6a0850832656"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_9a9fd15b-015b-4d71-aa34-ff3dbc8cd8c7" width="551" height="484"></a> </p> <p>&nbsp;</p> <p>The History view shows all the changes made, when and by who.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_65dd4408-f475-4605-8f94-06c93d4b412e"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_c82946c4-0fd1-4360-a05e-c420adcf69b1" width="551" height="484"></a> </p> <p>&nbsp;</p> <p>Clicking the Version number allows you to view the Step Configuration and restore it to the current version if you need to revert some changes you made.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_1d78078e-5a49-4a5f-99e6-5c0fa2b7bfa9"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_e65c5192-7d8e-430e-ae9d-bf46619fa586" width="551" height="484"></a></p> Team Blog Thu, 29 Aug 2013 14:51:25 +0000 http://www.simego.com/Blog/2013/08/Simego-Partner-XRM-Consultancy-Migrates-ACT--to-Dynamics-CRM-2011 info@simego.com (Sean Cleaver) Simego Partner - XRM Consultancy Migrates ACT! to Dynamics CRM 2011 <p>Our partner <a href="http://www.xrmconsultancy.com">XRM Consultancy</a> successfully migrates data from ACT! to Microsoft Dynamics CRM2011 with Data Synchronisation Studio. Read all about it <a href="http://www.xrmconsultancy.com/2013/07/migrating-from-act-to-dynamics-crm/" title="http://www.xrmconsultancy.com/2013/07/migrating-from-act-to-dynamics-crm/">http://www.xrmconsultancy.com/2013/07/migrating-from-act-to-dynamics-crm/</a></p> Team Blog Thu, 29 Aug 2013 14:40:30 +0000 http://www.simego.com/Blog/2013/08/Simego-Partner-ProTechnology-Data-islands-connected info@simego.com (Sean Cleaver) Simego Partner - ProTechnology Data islands connected <p>Simego Partner &ndash; <a href="http://www.protechnology.de/">ProTechnology GmbH</a> selects Simego Data Synchronisation Studio for client Data Integration requirements.</p> <p><a href="http://blog.protechnology.de/index.php/2012/09/dateninseln-verbinden-mit-simego-und-systemubergreifender-daten-integration/">http://blog.protechnology.de/index.php/2012/09/dateninseln-verbinden-mit-simego-und-systemubergreifender-daten-integration/</a></p> Team Blog Thu, 29 Aug 2013 13:55:48 +0000 http://www.simego.com/Blog/2013/08/Simego-Partner-P2-Automation-100--Improved-Data-Synchronisation info@simego.com (Sean Cleaver) Simego Partner - P2 Automation 100% Improved Data Synchronisation <p>Partner <a href="http://www.p2automation.com/">P2 Automation</a> selects Simego Data Synchronisation Studio for Data Integration read all about it.</p> <p><a href="http://www.p2automation.com/tips-tricks-a-tidbits/p2-automation-simego-100-improved-data-synchronization">http://www.p2automation.com/tips-tricks-a-tidbits/p2-automation-simego-100-improved-data-synchronization</a></p> Team Blog Thu, 29 Aug 2013 13:47:07 +0000 http://www.simego.com/Blog/2013/08/OUVVI--Update info@simego.com (Sean Cleaver) OUVVI: Update <p>We have a new release of Ouvvi available in our beta <a href="http://www.simego.com/Install/Ouvvi-Beta">download area</a> for our customers.</p> <p>&nbsp;</p> <p>This update to Ouvvi adds the Following new Features :-</p> <p>&nbsp;</p> <p>1. <strong>Documentation</strong> -&nbsp; With a Click of a button you can now generate full documentation detailing all the integration processes you have configured.</p> <p>2. <strong>Audit Events</strong> – We now log events from Users so when a user modifies a project/step it records the action in the Log.</p> <p>3. <strong>Change Control</strong> – Modifying a Step in Ouvvi now records this change in the Step History allowing for the user to recover old configurations by simply restoring the previous step.</p> Team Blog Thu, 15 Aug 2013 16:52:47 +0000 http://www.simego.com/Blog/2013/08/Formatting-Telephone-Numbers-in-DS3-using-Excel-Like-Calculated-Columns david@simego.com (David Evans) Formatting Telephone Numbers in DS3 using Calculated Column Expressions <p>Inspired by the posting on excellent PowerObjects blog called &lsquo;<a target="_blank" href="http://www.powerobjects.com/blog/2013/07/31/format-phone-numbers-via-scribe-insight-in-crm/">Format Phone Numbers with Leading Zeros via Scribe Insight in CRM</a>&rsquo; I wanted to demonstrate how to format a telephone number using Calculated columns.</p> <p>&nbsp;</p> <p>The challenge is to format a raw value in a field called into a telephone number if it is of the correct format. JoeCRM on PowerObjects defines it more clearly:</p> <p>&nbsp;</p> <p>&lsquo;The following function will not only handle the leading zeros, but if there are exactly 10 numeric characters in the source, it will also format it as (xxx) xxx-xxxx. If there are not exactly 10 characters in the source, the function will simply populate the raw value from your source.&rsquo;</p> <p>&nbsp;</p> <p>In Scribe Insight you would enter the following transformation:</p> <p>&nbsp;</p> <p>IF(ISERROR(S8),#NULL!,IF(S8=&rdquo;NULL&rdquo;,#NULL!,IF(LEN(STRIP(S8,&rdquo;N&rdquo;))=10,CONCATENATE(&ldquo;(&ldquo;,LEFT( STRIP(S8,&rdquo;N&rdquo;),3 ), &ldquo;) &ldquo;, MID( STRIP(S8,&rdquo;N&rdquo;),4 ,3 ),&rdquo;-&rdquo;,MID( STRIP(S8,&rdquo;N&rdquo;),7 ,4 )),LEFT(S8,30))))</p> <p>&nbsp;</p> <p><em>Note: the S8 refers to the field index but is not the field name.</em></p> <p><em>&nbsp;</em>&nbsp;</p> <p>Using DS3&rsquo;s Calculated Column Editor you would generate the following expression where the field is called targettext.</p> <p>Obviously commenting your expression as not to cause immense pain when coming back to it later:</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_43f7c5a2-4931-4a63-8c57-b356e0014b5a"><img height="501" width="663" src="https://simegoimages.blob.core.windows.net/images/blog_3904ee79-1697-4219-8370-b23d5616681b" alt="image" border="0" style="display: inline; border-width: 0px;" title="image" /></a></p> <p><em>&nbsp;</em>&nbsp;<em>Note: You can get a full list of Functions here: </em><a href="http://www.simego.com/Help/Functions" title="http://www.simego.com/Help/Functions">http://www.simego.com/Help/Functions</a></p> <h2>Calculated Columns</h2> <p>&nbsp;</p> <p>Calculated columns offer the following efficiency, productivity and quality benefits:</p> <ol> <li>The comments are preserved and appear in the Automatically generated documentation <p><a href="https://simegoimages.blob.core.windows.net/images/blog_886000fd-1f3f-4f5d-a16b-2ca3f78dcbed"><img height="258" width="659" src="https://simegoimages.blob.core.windows.net/images/blog_1e499c80-6ab5-4412-aa4a-ad84e0600879" alt="image" border="0" style="display: inline; border-width: 0px;" title="image" /></a></p> <p>&nbsp;</p> </li> <li>When putting together an expressions in Calculated Columns intelli-sense makes it easier and more accurate: <p><a href="https://simegoimages.blob.core.windows.net/images/blog_914b1e9a-64ae-4811-9031-d20ec343d6b6"><img height="332" width="661" src="https://simegoimages.blob.core.windows.net/images/blog_6b41ebc6-c0b9-48b3-ad96-4e177910713d" alt="image" border="0" style="display: inline; border-width: 0px;" title="image" /></a></p> <p>&nbsp;</p> </li> <li>If you need to refer to a source column you have full auto-complete of columns and expressions: <p><a href="https://simegoimages.blob.core.windows.net/images/blog_165b3001-dd7f-4265-9f28-e42ebd10615e"><img height="499" width="657" src="https://simegoimages.blob.core.windows.net/images/blog_2c52bb67-6c32-47c9-a876-0892c1bc5046" alt="image" border="0" style="display: inline; border-width: 0px;" title="image" /></a></p> </li> </ol> Team Blog Thu, 15 Aug 2013 13:13:07 +0000 http://www.simego.com/Blog/2013/06/Schema-Mapping-Improved info@simego.com (Sean Cleaver) Schema Mapping Improved <p>Great news we’ve re-worked the Schema Mapping to make things faster and more obvious.</p> <p>You now just click the checkbox against the columns you want in the schema map and then there automatically added and mapped to the target if possible.</p> <p>This also gives you a visual clue as to what columns have been mapped.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_6dd628e5-c21a-43c1-be9d-0e4ca6576da1"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_e3a00f0f-2e15-4e3b-9531-83c12e1a5bc0" width="644" height="415"></a></p> Team Blog Tue, 18 Jun 2013 16:27:27 +0000 http://www.simego.com/Blog/2013/06/Creating-a-Custom-Data-Source info@simego.com (Sean Cleaver) Creating a Custom Data Source <p>If you have a system that your trying to integrate that is not supported by Data Sync it is quite easy to create a custom Data Provider to read data from this system and return it to Data Sync.</p> <p>&nbsp;</p> <p>The First Step is to Connect Data Source A to the <strong>Empty/Dynamic Data source</strong> Provider. This provider does not return any rows but allows you to define the returned schema and override the row loading in Dynamic Columns.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_c5c4d0b8-bb10-42f0-8506-6b13c282dd99"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_558363ab-0c66-47df-98a4-002d9b6307c6" alt="image" style="display: inline; border-width: 0px;" title="image" height="356" width="644" /></a></p> <p>&nbsp;</p> <p>Define your Schema Columns within the Columns property window.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_a9e738ee-5175-4ef2-ac37-39b89280fe5d"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_79e2ee43-f56a-43b3-b7c5-26b0d6160049" alt="image" style="display: inline; border-width: 0px;" title="image" height="377" width="511" /></a></p> <p>&nbsp;</p> <p>When you preview the data in Data Sync you will get an empty result</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_092d3852-2d4d-46c9-a466-82fa96daea67"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_272ad925-5797-4061-ae35-8fbcdaf7c3a9" alt="image" style="display: inline; border-width: 0px;" title="image" height="253" width="644" /></a></p> <p>&nbsp;</p> <p>However switching to the Dynamic Columns view you can override the <strong>GetDataTable</strong> method to provide your own Row Loading implementation and by using this you can integrate with virtually anything you like.</p> <p>The code below&nbsp;is what&nbsp;you would require to implement your own data source.</p> <p>This code simply returns ID and Name columns but it could easily be expanded for any other use.</p> <pre class="csharpcode"><span class="preproc">#region</span> Usings <span class="kwrd">using</span> System; <span class="kwrd">using</span> System.Collections.Generic; <span class="kwrd">using</span> System.Data; <span class="kwrd">using</span> System.Globalization; <span class="kwrd">using</span> System.Text; <span class="kwrd">using</span> System.Linq; <span class="kwrd">using</span> Simego.DataSync; <span class="preproc">#endregion</span> <span class="kwrd">using</span> Simego.DataSync.Providers.Null; <span class="rem">/// &lt;summary&gt;</span> <span class="rem">/// This module of code allows you to define some columns in code or Dynamic Columns</span> <span class="rem">/// To Create new Dynamic Columns simply expose them as properties in this class.</span> <span class="rem">/// &lt;/summary&gt;</span> <span class="kwrd">partial</span> <span class="kwrd">class</span> DataSourceRowOverride : Simego.DataSync.DynamicColumns.DataSourceRowInternal <span class="rem">//Do Not Change This Line</span> { <span class="kwrd">public</span> <span class="kwrd">override</span> DataTableStore GetDataTable(DataTableStore dt, IDataSourceReader reader) { <span class="rem">//Get the Actual Reader Instance.</span> NullDataSourceReader SourceReader = (NullDataSourceReader)reader; <span class="rem">//Create a Schema Mapping Helper</span> DataSchemaMapping mapping = <span class="kwrd">new</span> DataSchemaMapping(reader.SchemaMap, reader.Side); <span class="kwrd">for</span>(<span class="kwrd">int</span> i=0; i&lt;1000; i++) { var row = dt.NewRow(); <span class="kwrd">foreach</span> (DataSchemaItem item <span class="kwrd">in</span> reader.SchemaMap.GetIncludedColumns()) { <span class="kwrd">string</span> columnName = mapping.MapColumnToDestination(item); <span class="kwrd">switch</span>(columnName) { <span class="kwrd">case</span> <span class="str">"ID"</span>: { row[item.ColumnName] = DataSchemaTypeConverter.ConvertTo(i, item.DataType); <span class="kwrd">break</span>; } <span class="kwrd">case</span> <span class="str">"Name"</span>: { row[item.ColumnName] = DataSchemaTypeConverter.ConvertTo(<span class="kwrd">string</span>.Format(<span class="str">"Item-{0}"</span>, i), item.DataType); <span class="kwrd">break</span>; } <span class="kwrd">default</span>: { <span class="kwrd">break</span>; } } } <span class="kwrd">if</span> (dt.Rows.Add(row) == DataTableStore.ABORT) <span class="kwrd">break</span>; } <span class="kwrd">return</span> dt; } } </pre> <p>&nbsp;</p> <p>Now when you preview the data you will get data from your source system which you can then connect to a Target system just like any other provider.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_810dd29c-26bd-402d-8a50-5988f9ab4b64"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_3a63168b-cc4b-4957-a70d-1f4e788c2f6e" alt="image" style="display: inline; border: 0px;" title="image" height="315" width="644" /></a></p> Team Blog Mon, 17 Jun 2013 11:07:36 +0000 http://www.simego.com/Blog/2013/06/Import-Multiple-Xml-Documents info@simego.com (Sean Cleaver) Import Multiple Xml Documents <p>This is an example of how to use Data Synchronisation Studio to import multiple Xml Documents from a Folder.</p> <p>Imagine you have a process where Xml Documents are dropped into a Folder each one being a Single Record in your target system. (The Documents could contain multiple Items however this example uses a Single Document per Record)</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_e370a601-56bf-46d8-89a4-3a20148e4ede"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_a8f1f917-c353-4cf6-9fc8-0e7f64aa0243" width="455" height="484"></a> </p> <p>&nbsp;</p> <p>These Xml documents could be complex however for this example we’re going to use something simple</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_529ddbba-6dbb-46fc-84a6-88595157ee1e"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_0b412197-5a5d-49f7-a3f0-9571c17d4133" width="287" height="178"></a> </p> <p>&nbsp;</p> <p>Using the Xml File Provider in Data Sync this import is very easy.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_51520c88-df91-44d8-9b02-dedca0cdb041"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_d44d5740-d038-4ae5-aa65-513d8aa73983" width="644" height="356"></a> </p> <p>&nbsp;</p> <p>First set the Document Item Element XPath to <strong>/Document</strong></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_931a6915-6307-4aaf-8aec-ad2db92256b3"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_b0ad6cee-7d25-4d00-9054-3fbf89b5b0d0" width="387" height="169"></a> </p> <p>Then Define the Columns that you want to return from the Xml Document <strong>ID</strong> and <strong>Name</strong> in this Example.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ed1e50af-471f-4701-aa65-0874121b3311"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_49c6fdf4-9572-4956-9c47-625ac15385b6" width="511" height="377"></a>&nbsp;</p> <p>&nbsp;</p> <p>Then enter the path to the XML Documents as a Wildcard Path i.e.<strong> C:\Temp\Import\*.xml</strong></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_fe30f78b-829f-4bd5-9537-12f744dee301"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_978d11bc-1ef2-4d2b-9f19-f4d6d0b52ffb" width="644" height="356"></a> </p> <p>&nbsp;</p> <p>It’s that easy, Data Sync will then enumerate all the Xml Documents that are returned from the Wildcard path and return the result as a single result that you can then import/synchronise with your Target System i.e. SQL Server, CRM etc.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_473d1862-2fdc-40f0-9c74-28f507cbc54c"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_2ff5ccff-936e-4637-b4e8-fef3e7136841" width="248" height="484"></a></p> Team Blog Mon, 17 Jun 2013 10:51:16 +0000 http://www.simego.com/Blog/2013/06/How-to-integrate-with-Case--indicent--in-Dynamics-CRM-2011 david@simego.com (David Evans) How to integrate with Case (incident) in Dynamics CRM 2011 <p><em>Note: The Dynamics CRM User Interface uses the terminology &lsquo;Cases&rsquo; but the underlying system calls them incidents</em></p> <p>Dynamics CRM is an excellent choice when managing service contracts but integrating the services process with other systems can be difficult.</p> <p>Essentially, the life-cycle of an case (incident) is:</p> <ol> <li>Create the incident from an account or contact </li> <li>Add several activities to the incident </li> <li>Update the incident setting the status to closed / resolved</li> </ol> <p>The above can be achieved from the Dynamics CRM user interface simply but the underlying architecture or how the incident moves between states is not quite as simple.</p> <p>In a perfect world we would assume that the following could easily be achieved by calling the following pseudo SDK calls:</p> <ol> <li>Create Incident </li> <li>Create Activities and associate them with the incident </li> <li>Create a Close Incident Activity </li> </ol> <p>Unfortunately, we need to apply some more details and steps to the solution rather than the standard Create, Update, Delete state changes:</p> <ol> <li>Create Incident </li> <li>Create Activities and associate them with the incident </li> <li>Close Incident by sending specific Close Incident Message</li> </ol> <p>Many other entities in Dynamics CRM 2011 have the same pattern of messages to change the state of the entity.</p> <p>Fortunately, Data Sync Studio will just call the special messages for you, so you can easily integrate you cases and incidents with &lsquo;no code&rsquo;, just using the simple click-click user interface and some excel type functions.</p> <h2>Using Data Synchronisation Studio to Create and Close Dynamics CRM Incidents with no code</h2> <p>&nbsp;</p> <h3>Step 1. Connect to your source Service Desk system</h3> <p>At Simego we have an external ticketing system connected to the web, twitter, email etc. We are integrating this with Dynamics CRM and need to created historic as well as new tickets (incidents).</p> <p>We connect to our Service Desk SQL Database</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d5858eb7-9be8-47d3-831b-0d292d0a25da"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_708772e7-e4b9-47aa-a07c-4b16d8f1d388" alt="image" style="background-image: none; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border: 0px;" title="image" height="365" width="656" /></a></p> <p>Browse through to the Tickets Table (we can just as easily define a SQL query as source)</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_6b7b5290-8943-414c-8554-975fc4ce6536"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_9a569290-6e18-4031-873b-99c3c3ed34f3" alt="image" style="background-image: none; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border: 0px;" title="image" height="491" width="489" /></a></p> <h3>Step 2. Connect to the target Dynamics CRM 2011 incident entity</h3> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_5815c1dd-84a3-4f2d-8f4c-b7bd0ceb09f9"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_b58f5e26-ee6c-4ba6-8bfd-8d77c8bc32c2" alt="image" style="background-image: none; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border: 0px;" title="image" height="368" width="660" /></a></p> <h3>Step 3. Map the required columns and use some Excel like functions to create the correct status codes and meta-data.</h3> <p>When creating an incident in Dynamics CRM you need to specify a contact or account to associate the incident to. We can use a &lsquo;Calculated Column&rsquo; to quickly lookup the contactid from the contact entity based on a userid of the users stored externaluseridentifier column:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_9956b8e6-de2e-4042-96e7-77775d414521"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_918b13fc-236e-4dc2-aef7-49666f8f8703" alt="image" style="background-image: none; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border: 0px;" title="image" height="361" width="621" /></a></p> <p>We also can calculate the correct status mapping from the source system to status codes in Dynamics CRM. In this example is the Service Desk ticket status is 0 (open) then the statuscode of the incident needs to be set to 1 (&lsquo;In Progress&rsquo;)else set the code to 5 (&lsquo;Problem Solved&rsquo;).</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_b7dc35c6-48f7-48fe-b17e-1bf4a3213b77"><img border="0" src="https://simegoimages.blob.core.windows.net/images/blog_15918e49-9c31-466d-8df9-b328c0a28644" alt="image" style="background-image: none; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border: 0px;" title="image" height="463" width="660" /></a></p> <h3>Step 4. Compare and synchronise</h3> <p>In this step DS3 calculates the differences between the Ticket table and the Incident entity, calculates a reconciliation and generates the Dynamic CRM inserts, Updates and Deletes.</p> <p>More importantly when we update the status of the incident (case) to closed it will automatically send the correct CloseIncidentRequest&rsquo;s to close the request.</p> <p>This allows our client&rsquo;s to choose when they close their tickets rather than our support team. This increases customer satisfaction as the support team cannot assume that an issue has been resolved satisfactorily. The client is in control of the call closure.</p> Team Blog Wed, 05 Jun 2013 13:46:26 +0000 http://www.simego.com/Blog/2013/06/Creating-Dynamics-CRM-Phonecall-Activities info@simego.com (Sean Cleaver) Creating Dynamics CRM Phonecall Activities <p>To create activity records in dynamics CRM 2011 with Data Sync requires a little bit of data formatting to link the appropriate entity and user.</p> <p>&nbsp;</p> <p>We have a special format that is used to identify the linked entity which consists of an entity name followed by | then the guid of the item in CRM.</p> <p>&nbsp;</p> <p><strong>For Example</strong></p> <p><strong><br /></strong></p> <p>A lead entity might look like this</p> <p><span style="text-decoration: underline;">lead|8bae1373-5ac4-e211-9980-002170572d07</span></p> <p><span style="text-decoration: underline;"><br /></span></p> <p>Then it&rsquo;s a simple matter of formatting your source data appropriately i.e. below is an example of data you might supply to the phonecall entity to create phonecall activity records.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_1121e1f9-c4f4-4219-bddd-fb1b1df2e255"><img title="image" style="display: inline; border-width: 0px;" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_3025da68-571c-46b6-a743-dbae7374a726" width="644" height="73" /></a></p> <p>&nbsp;</p> <p>The Schema map might look like this.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_26cc1f61-deee-4b7c-b1c8-3234ec1a1766"><img title="image" style="display: inline; border-width: 0px;" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_5b1b7677-30a2-4a4a-b8c0-2509542cde89" width="644" height="240" /></a></p> <p>&nbsp;</p> <p>You will notice that the to and from fields are data type <em>System.String []</em>&nbsp; this is because these can contain multiple values. If you need to supply multiple values just separate them with a semi-colon (;).</p> <p>&nbsp;</p> <p>account|494ABAFA-3B2B-4DC2-872F-EB5FDF0B9F39;contact|35622FD8-6620-470C-BEDE-46A4E759F050</p> Team Blog Tue, 04 Jun 2013 13:55:33 +0000 http://www.simego.com/Blog/2013/06/Migrate-documents-on-file-system-to-Microsoft-Dynamics-CRM-2011-as-attachments info@simego.com (Sean Cleaver) Migrate documents on file system to Microsoft Dynamics CRM 2011 as attachments <p>This is an example of how you can use Data Sync to migrate/import documents to Microsoft Dynamics CRM.</p> <p>&nbsp;</p> <p>This example assumes you have your files arranged into Folders where the Folder name is the name of the Account that you wish to link the file to.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_423c1ab2-ebfb-4056-ac58-870ea34ad2ea"><img title="image" style="display: inline; border: 0px;" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_cb5df11f-743b-43ce-b520-564b31d23e20" width="513" height="221" /></a></p> <p>&nbsp;</p> <p>Step 1 &ndash; Connect the File System Provider to the Root path that contains your Files.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_5c530083-3528-433d-97dc-644680f60b8f"><img title="image" style="display: inline; border: 0px;" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_0f393a7d-bb57-43b5-bd71-984f1bd90f2a" width="644" height="356" /></a></p> <p>&nbsp;</p> <p>Step 2 &ndash; Create Calculated Columns to Return Missing Metadata.</p> <p>&nbsp;</p> <p>CRM Requires that the File data is passed as a BASE64 encoded string, simple create an Expression to convert the File data to Base64 like this.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_de40b205-9fdf-4a96-8afd-c89c567c3a6c"><img title="image" style="display: inline; border: 0px;" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_5a3a859c-d4ba-49b4-9ea8-e650f2ef74a6" width="644" height="286" /></a></p> <p>&nbsp;</p> <p>We need an Object ID Type code of account so we just return the string &ldquo;account&rdquo;</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_193bc0d8-36d5-4a97-a206-53e211ad3a91"><img title="image" style="display: inline; border: 0px;" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_902dbf39-fe18-4dde-9fd5-791d4fc088d1" width="456" height="331" /></a></p> <p>&nbsp;</p> <p>Finally we need to map the path to the account name with a Lookup function.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_a7db867f-2c8d-4dc8-acda-9088e9e3d67b"><img title="image" style="display: inline; border: 0px;" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_585fa14f-9acc-4dfe-9bec-f3763b3e875e" width="456" height="331" /></a></p> <p>&nbsp;</p> <p>Step 3 &ndash; Connect Target to CRM</p> <p>&nbsp;</p> <p>Then Connect your Target Data source B to the CRM Annotation Entity and map like this.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_c09fa755-bb45-4fcb-800d-e7f78ece6ec0"><img title="image" style="display: inline; border: 0px;" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_83aaea3e-d5ef-49fc-9879-3b0390d2233c" width="644" height="190" /></a></p> <p>&nbsp;</p> <p>That&rsquo;s it just compare and sync for a true No-Code (kinda no code anyway) solution :-)</p> Team Blog Tue, 04 Jun 2013 11:28:45 +0000 http://www.simego.com/Blog/2013/06/Lookup-Functions-Everywhere info@simego.com (Sean Cleaver) Lookup Functions Everywhere <p>We’ve revamped the lookup function capability in our next release so support many more lookup function scenarios. The new Lookup functions make it really quick and easy to lookup a value from a remote system based on a value from your source.</p> <p>The Functions are :-</p> <p><strong>LOOKUPA</strong> -&nbsp; This function uses the Datasource A system details to lookup a value. For example if your Datasource A is SQL Server then this lookup will use the same connection string details but allow you to override the Table to look in.</p> <p>For example the following Expression</p> <p>LOOKUP <strong>CompanyName</strong> FROM <strong>Suppliers</strong> WHERE <strong>SupplierID</strong> EQUALS <strong>SupplierID</strong></p> <p>Is written as a Calculated Column Function Like this</p> <p>LOOKUPA("CompanyName", "Suppliers", KV("SupplierID", SupplierID))</p> <p>This is entered into a Data Sync Calculated Column like this.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_517ebe5e-1ba4-4232-9ed4-2cb438c255c9"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_8bb730a9-322d-4afd-ae74-9ab834496b54" width="456" height="331"></a> </p> <p>The KV function is a Key-Value function that is used to pass parameters to multiple functions.</p> <p>Expanding on this function you can override the Data source Properties with a FROM parameter i.e you can pass in your own SQL Query through a KV Function. i.e. </p> <p>LOOKUPA("CompanyName", <strong>FROM(KV("Command", "SELECT SupplierID, CompanyName FROM Suppliers"))</strong>, KV("SupplierID", SupplierID))</p> <p>You can also chain the conditions where you need to match on multiple values. i.e.</p> <p>LOOKUPA("CompanyName", "Suppliers", KV("SupplierID", SupplierID), <strong>KV(“Country”, “UK”)</strong>)</p> <p>&nbsp;</p> <p><strong>LOOKUPB</strong> – This is functionally equivalent to LOOKUPA except that it uses your Data source B (Target) connection details.</p> <p>For example where your target is Dynamics CRM you can write a Lookup Expression to lookup the AccountID from the account Entity where the names match like this.</p> <p>LOOKUPB(“accountid”, “account”, KV(“name”, name))</p> <p>&nbsp;</p> <p><strong>LOOKUPEX</strong> – This function can be used for simple lookups against a CSV, Excel or Data Sync Project.</p> <p>To lookup from an Excel Spreadsheet you can use this Function</p> <p>LOOKUPEX("Name", <strong>FROMEXCEL</strong>("C:\\Users\\Sean\\Documents\\Numbers.xlsx"), KV("ID", ID))</p> <p>To Lookup from a CSV File you can use this Function</p> <p>LOOKUPEX("Name", <strong>FROMCSV</strong>("C:\\Users\\Sean\\Documents\\Numbers.csv"), KV("ID", ID))</p> <p>And finally to use an Existing Data Sync project that is connected to any other data source you can use this function.</p> <p>LOOKUPEX("Name", <strong>FROMDATASYNC</strong>("C:\\Users\\Sean\\Documents\\Numbers.dsprj"), KV("ID", ID))</p> Team Blog Tue, 04 Jun 2013 10:42:59 +0000 http://www.simego.com/Blog/2013/05/Calculated-Columns-UI-Enhancements info@simego.com (Sean Cleaver) Calculated Columns UI Enhancements <p>Just a quick post to show that we now have code highlighting and intellisense type support for our new calculated column feature.</p> <p>&nbsp;</p> <p>For example below is a screenshot of our CASE function which can be used to transform a value into 1 of several possibilities.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_8d9180c7-2990-4022-ac91-44e577b4ada2"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_b4836ef6-ce49-47d8-a829-d3ff03b1bc8e" width="637" height="396"></a> </p> <p>&nbsp;</p> <p>We also have a full list of our supported functions here <a href="http://www.simego.com/Help/Functions">http://www.simego.com/Help/Functions</a></p> Team Blog Mon, 20 May 2013 15:36:56 +0000 http://www.simego.com/Blog/2013/05/Calculated-Columns info@simego.com (Sean Cleaver) Calculated Columns <p>We have a great new feature for Data Sync that we just have to tell you about, previously if you wanted to transform some value we had &ldquo;Dynamic Columns&rdquo; where you could write your own .NET C#. Now we have a simplified yet very powerful solution for when you just need something simple or if your not very good with .NET. We&rsquo;ve tried to make it similar to other scripting solutions whilst keeping the ability to be really advanced if you need to.</p> <p>For example below is a little snippet that adds 30 days to Today.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_e488cbf5-e2a7-402c-97b4-1fd8755ef33a"><img height="331" width="410" src="https://simegoimages.blob.core.windows.net/images/blog_05d32338-0c2a-412e-877f-3719adc31ae1" alt="image" border="0" style="display: inline; border-width: 0px;" title="image" /></a></p> <p>As with everything we do we&rsquo;ve made it really easy just hit the (Fx) button on your Data Source to add a new column based on your Function.</p> <p>We&rsquo;ve added lots of useful functions so you can virtually do anything you need, plus you can even mix it with the existing Dynamic Columns feature.</p> <p>Here&rsquo;s a List of Current Functions with more on the way!</p> <p>AND(params bool[])<br />BITWISEAND(params int[])<br />BITWISECOMPLEMENT(params int[])<br />BITWISEOR(params int[])<br />BITWISEXOR(params int[])<br />CHOOSE(int?, params string[])<br />CHOOSE(int?, params int?[])<br />CHOOSE(int?, params double?[])<br />CHOOSE(int?, params object[])<br />CONCAT(params string[])<br />CONCATENATE(params string[])<br />DATALENGTH(byte[])<br />DATEVALUE(string)<br />DAY(System.DateTime?)<br />FALSE()<br />FROMBASE64(string)<br />HOUR(System.DateTime?)<br />IF(bool, object, object)<br />IF(bool, string, string)<br />IF(bool, int, int)<br />INT(object)<br />ISINT(string)<br />ISNULL(object)<br />ISNULLOREMPTY(string)<br />ISNUMBER(string)<br />ISO8601DATE(System.DateTime?)<br />ISO8601DATETIME(System.DateTime?)<br />JOIN(string, params string[])<br />LEFT(string, int)<br />LEN(string)<br />LENGTH(string)<br />LOWER(string)<br />LTRIM(string)<br />MID(string, int, int)<br />MILLISECOND(System.DateTime?)<br />MINUTE(System.DateTime?)<br />MOD(int?, int)<br />MONTH(System.DateTime?)<br />NEWGUID()<br />NOT(bool)<br />NOW()<br />NULL()<br />NUMBER(object)<br />OR(params bool[])<br />PROPER(string)<br />REPLACE(string, string, string)<br />RIGHT(string, int)<br />RND()<br />RTRIM(string)<br />SEARCH(string, string)<br />SEARCH(string, string, int)<br />SECOND(System.DateTime?)<br />SPLIT(string, string)<br />STR(object)<br />TOBASE64(byte[])<br />TODAY()<br />TRIM(string)<br />TRUE()<br />UPPER(string)<br />UTCNOW()<br />WEEKDAY(System.DateTime?)<br />WEEKNUM(System.DateTime?)<br />YEAR(System.DateTime?)</p> Team Blog Wed, 15 May 2013 20:54:26 +0000 http://www.simego.com/Blog/2013/05/How-to-insert-a-large-amount-of-records-into-Dynamics-CRM-Online david@simego.com (David Evans) How to insert a large amount of records into Dynamics CRM Online <p>&nbsp;</p> <p><strong>Thoroughly updated for the changes in Dynamics CRM Online (2013) on 07-July-2014. You can be lazy and skip to a demonstration video at the end of the post.</strong></p> <p>&nbsp;</p> <p>As more and more value is being derived from enhancing Dynamics CRM with integrated data, there has been an issue with the rate at which you can create and then maintain records with in your Dynamics CRM online environment. It is considered slow, expensive and difficult to achieve, this is more down to the implementations of the third party vendors rather than some failing of the infrastructure or Dynamics API design.</p> <p>&nbsp;</p> <p>For this demonstration we are going to use a default Dynamics Online instance with no optimisation, I did not need:</p> <ol> <li>Microsoft Server Products </li> <li>Microsoft SQL Server Products including DTS or SSIS </li> <li>Any type of message queuing </li> <li>Any type of xml experience. </li> </ol> <p>&nbsp;</p> <p>I am going to import 50000 simple contact records into the contact entity in a Dynamics CRM Online instance but this solution scales directly to 1,000,000s records .</p> <p>&nbsp;</p> <p>I did require:</p> <ol> <li>The 50000 contact file can be found here (<a href="http://download.simego.com/files/Sample-Data/50000-Sample-Contacts.zip" title="http://download.simego.com/files/Sample-Data/50000-Sample-Contacts.zip">http://download.simego.com/files/Sample-Data/50000-Sample-Contacts.zip</a>). </li> <li>Data Synchronisation Studio v3 found here (<a href="http://www.simego.com/Products/Data-Synchronisation-Studio" title="http://www.simego.com/Products/Data-Synchronisation-Studio">http://www.simego.com/Products/Data-Synchronisation-Studio</a>)&nbsp; </li> </ol> <p>&nbsp;</p> <p>I connected the Source to the CSV file and selected the 50000 record contact file.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_65423b58-5be9-449b-aeab-c2078e1d6a31"><img height="504" width="915" src="https://simegoimages.blob.core.windows.net/images/blog_ccc78e13-3984-4f70-b677-5f281b7418e9" alt="Connecting to a CSV FIle to Importing Into Dynamics CRM" style="display: inline; border: 0px;" title="Connecting to a CSV FIle to Importing Into Dynamics CRM" /></a></p> <p>&nbsp;</p> <p>I then connected the destination to our dynamics CRM Online instance.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_e9a04034-94e0-4e6f-beeb-2d3341ee98e1"><img height="504" width="915" src="https://simegoimages.blob.core.windows.net/images/blog_f93871a3-526e-43f3-8f56-8f282665ae62" alt="Connecting To Dynamics CRM to Import a CSV File" style="display: inline; border: 0px;" title="Connecting To Dynamics CRM to Import a CSV File" /></a></p> <p>&nbsp;</p> <p>I mapped contact and address details to make sure the sync reflected a realistic insert.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_21219360-15c5-4652-a61b-aae338e5841f"><img height="603" width="897" src="https://simegoimages.blob.core.windows.net/images/blog_e894bd68-0685-4d4d-93ce-2947ebfd9179" alt="Mapping between Dynamics CRM and a CSV FIle" style="padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border: 0px; background-image: none;" title="Mapping between Dynamics CRM and a CSV FIle" /></a></p> <p>&nbsp;</p> <p>The compare only takes about 2 seconds and DS3 predicts 50000 inserts and previews the records:</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_37a5d179-16ce-481a-800e-53b6282b69f0"><img height="894" width="1344" src="https://simegoimages.blob.core.windows.net/images/blog_fd57c549-c67a-4d30-930d-a2c3e4f04502" alt="Transforming Data during Dynamics CRM Integration " style="display: inline; border: 0px;" title="Transforming Data during Dynamics CRM Integration " /></a></p> <p>&nbsp;</p> <p>We have conducted extensive research in insert and update performance and we know that the current optimum Update Batch Size and parallelism in Dynamics CRM online is 16 Threads with 8 records per batch. This may seem counter intuitive but all of the performance testing has borne out these numbers.</p> <p>&nbsp;</p> <p><a href="http://www.simego.com/Blog/2013/10/Dynamics-CRM-Online-2011-to-2013-API-Performance" title="http://www.simego.com/Blog/2013/10/Dynamics-CRM-Online-2011-to-2013-API-Performance">http://www.simego.com/Blog/2013/10/Dynamics-CRM-Online-2011-to-2013-API-Performance</a></p> <p>&nbsp;</p> <p>It is very easy to introduce parallelism in to DS3 just change the &lsquo;ThreadCount&rsquo; value. As with the parallelism the batch size is also easily configurable using the &lsquo;UpdateBatchSize&rsquo; setting.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_8dd28b01-be48-488f-8c22-4d6b0913b3a0"><img height="203" width="331" src="https://simegoimages.blob.core.windows.net/images/blog_19250330-7609-48bd-8749-d6ccc654306e" alt="Configuring Batch Update and Parallelism in Dynamics CRM API" style="display: inline; border: 0px;" title="Configuring Batch Update and Parallelism in Dynamics CRM API" /></a>&nbsp;</p> <p>&nbsp;</p> <p>Clicking the synchronise button synchronises the 50000 records from the Source to the</p> <p>Destination. Our test take in 2 minutes and four seconds giving 400 records / second.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_af5f2928-3c09-487b-8864-1eb74fea63af"><img height="587" width="838" src="https://simegoimages.blob.core.windows.net/images/blog_b40edd04-04e3-44df-9410-aaa61c47beb7" alt="insert a large amount of records into Dynamics CRM Online" style="display: inline; border: 0px;" title="insert a large amount of records into Dynamics CRM Online" /></a></p> <p>&nbsp;</p> <p>The excellent team at PowerObjects recently did some performance testing recently using SSIS so we have some similar statistics:</p> <p>&nbsp;</p> <p><a href="http://www.powerobjects.com/blog/2014/06/18/crm-data-integration-performance-testing-dynamics-crm/" title="http://www.powerobjects.com/blog/2014/06/18/crm-data-integration-performance-testing-dynamics-crm/">http://www.powerobjects.com/blog/2014/06/18/crm-data-integration-performance-testing-dynamics-crm/</a></p> <p>&nbsp;</p> <table border="1" width="680" cellpadding="2" cellspacing="0"> <tbody> <tr> <td width="312" valign="top"><strong>Tool</strong></td> <td width="122" valign="top"><strong>Batch Size</strong></td> <td width="120" valign="top"><strong>Parallel</strong></td> <td width="124" valign="top"><strong>Records/hour</strong></td> </tr> <tr> <td width="312" valign="top">SSIS with KingsWaySoft Adapter</td> <td width="122" valign="top">1000</td> <td width="120" valign="top">2</td> <td width="124" valign="top">500k</td> </tr> <tr> <td width="312" valign="top">Scribe Online</td> <td width="122" valign="top">1000</td> <td width="120" valign="top">1</td> <td width="124" valign="top">200-300k</td> </tr> <tr> <td width="312" valign="top"><strong>Simego DS3</strong></td> <td width="122" valign="top"><strong>10</strong></td> <td width="120" valign="top"><strong>16</strong></td> <td width="124" valign="top"><strong>1440k</strong></td> </tr> </tbody> </table> <p>&nbsp;</p> <p>These are indicative numbers and you should conduct your own testing.</p> <p>&nbsp;</p> <p>This in itself is faster than all competing solutions but what is more important is the ability to transform the source system data and re-synchronise. This is where a significant amount of migration pain is felt.</p> <p>&nbsp;</p> <p>For example we could decide that the email address formatting is completely incorrect and now everyone has got the own domain names. So James Burns has an email address of <a href="mailto:James@burns.com">James@burns.com</a>.</p> <p>&nbsp;</p> <p>We can achieve this by using a calculated column which is like an excel function (in this case we used the CONCATENATE function). This just creates a new column in the source.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_472897d4-6744-4b26-a994-a6c91d44cf07"><img height="404" width="537" src="https://simegoimages.blob.core.windows.net/images/blog_fb2a44b6-4dfd-48c2-b862-2fec56301e39" alt="Simple Dynamics CRM Integration Transformations in Simego DS3" style="display: inline; border: 0px;" title="Simple Dynamics CRM Integration Transformations in Simego DS3" /></a></p> <p>&nbsp;</p> <p>I can then just map this new field onto the email in the entity.</p> <p>&nbsp;</p> <p>Comparing the 50000 locally and the 50000 on Dynamics CRM Online takes a maximum of a couple of seconds. DS3 then shows you what it will be updated within Dynamics online. It will ONLY update columns that have been changed, meaning it will not update other fields which may trigger workflows or plug-ins inadvertently. Previewing the changes reduces any expensive rework in mapping and transformations.</p> <p>&nbsp;</p> <p><a target="_blank" href="https://simegoimages.blob.core.windows.net/images/blog_54d0ad67-063d-4bc0-ab93-032d30f4252e"><img height="917" width="1380" src="https://simegoimages.blob.core.windows.net/images/blog_724ba314-19b9-436b-bb4c-2e5c0b483c24" alt="Preview of Update to Data before Changes" style="display: inline; border: 0px;" title="Preview of Update to Data before Changes" /></a></p> <p>&nbsp;</p> <p>This update takes a nearly exact time of 2 minutes and 6 seconds again giving an update performance of over 1.4 Million updates per hour.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d6850069-3e78-44f8-901e-2b868fc5de3a"><img height="440" width="613" src="https://simegoimages.blob.core.windows.net/images/blog_3a30f690-9c15-457a-a04f-742f2007d0bd" alt="High performance updates in Dynamics CRM Online" style="display: inline; border: 0px;" title="High performance updates in Dynamics CRM Online" /></a></p> <p>&nbsp;</p> <p>Whilst it is possible for other products to use parallel servers with message queues, or control of flow components, it can soon becomes, expensive, unmanageable and time consuming without matching DS3 performance.</p> <p>&nbsp;</p> <p>If you would like a quick demonstration of DS3's Dynamics CRM integration (or any other Sharepoint, Active Directory integration) please feel free to drop an email to <a href="mailto:support@simego.com">support@simego.com</a>.</p> <p>&nbsp;</p> <p>We really are always happy to show of our solutions, share some of our clients successes and discuss Dynamics CRM API performance.</p> <p>&nbsp;</p> <p><strong>Here is the accompanying video demonstrating the difference in performance of many threads with small batches.</strong></p> <p>&nbsp;</p> <iframe height="676" width="900" name="wistia_embed" class="wistia_embed" scrolling="no" frameborder="0" allowtransparency="true" src="http://feeds.simego.com//fast.wistia.net/embed/iframe/z9n9zip6c0?videoFoam=true"></iframe> <script type="text/javascript"></script> Team Blog Mon, 13 May 2013 15:09:45 +0000 http://www.simego.com/Blog/2013/03/Exporting-Data-from-Podio-into-SQL-Server info@simego.com (Sean Cleaver) Exporting Data from Podio into SQL Server <p>Here’s an example where were using Podio to allow users to enter details on forms upload pictures and then use Data Sync to integrate this data with SQL Server.</p> <p>Data Sync will Synchronise the Form Data with a SQL Database and download the attached files into a Folder on the file system.</p> <p>&nbsp;</p> <p>Here’s a typical form in Podio and we want to export this Meta Data to SQL and download the picture files.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_c785f336-0e69-4f0c-a4d4-42bbf4bd4b80"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_149d7693-f441-4f69-bf1f-790a6c3d0e8f" width="643" height="484"></a> </p> <p>&nbsp;</p> <p>First we connect Data Sync to this App in Podio and include the columns were interested in the schema map.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_a26ce5ba-d88a-4d7c-9ae0-840aedbf9117"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_3201f716-0882-4dff-8299-d72c9cf0e46f" width="644" height="454"></a> </p> <p>&nbsp;</p> <p>Using the Tools Create SQL Table Menu we automatically create a SQL Table from this Schema to import the Data.</p> <p>Now everything is Mapped as we Expect.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_64628ac4-1fb1-4ac4-9fa5-bebe87db8411"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_c76d90bd-d75a-4e42-b590-ad3a9a5788db" width="644" height="370"></a></p> <p>&nbsp;</p> <p>The Next Step is to use the Project automation feature of Data Sync to add your own code to download the files from Podio and write them to the Local Disk. </p> <p>The Podio Provider exposes a special “Podio” object that allows us to interact directly with the Podio API, we added a special GetFile Method to this object to make it easy to download files from Podio.</p> <p>At the End of the Synchronisation process we iterate through the New items collection and get each file and write them to the disk with the code below.</p> <p>&nbsp;</p><pre class="csharpcode"><span class="preproc">#region</span> Usings <span class="kwrd">using</span> System; <span class="kwrd">using</span> System.Collections.Generic; <span class="kwrd">using</span> System.Data; <span class="kwrd">using</span> System.Globalization; <span class="kwrd">using</span> System.Text; <span class="kwrd">using</span> System.Linq; <span class="kwrd">using</span> Simego.DataSync; <span class="preproc">#endregion</span> <span class="kwrd">using</span> System.IO; <span class="rem">/// &lt;summary&gt;</span> <span class="rem">/// &lt;/summary&gt;</span> <span class="kwrd">class</span> ProjectAutomationOverride : Simego.DataSync.Automation.ProjectAutomationShim <span class="rem">//Do Not Change This Line</span> { <span class="kwrd">public</span> <span class="kwrd">override</span> <span class="kwrd">void</span> Start() { } <span class="kwrd">public</span> <span class="kwrd">override</span> <span class="kwrd">void</span> BeforeCompare(ProjectAutomationLoadResult loadResult) { } <span class="kwrd">public</span> <span class="kwrd">override</span> <span class="kwrd">void</span> BeforeSynchronisation(ProjectAutomationCompareResult compareResult) { } <span class="kwrd">public</span> <span class="kwrd">override</span> <span class="kwrd">void</span> End(ProjectAutomationResult result) { <span class="kwrd">if</span> ( result.HasChanges ) { <span class="kwrd">foreach</span>(var item <span class="kwrd">in</span> result.CompareResult.New) { <span class="kwrd">string</span> path = Path.Combine( <span class="str">@"C:\Temp\PodioExport"</span>, item.SourceRow[5].BeforeColumnValue.ToString()); File.WriteAllBytes(path, DataSourceA.Podio.GetFile(item.SourceRow[6].BeforeColumnValue.ToString())); } } } <span class="kwrd">public</span> <span class="kwrd">override</span> <span class="kwrd">void</span> Error(ProjectAutomationException e) { } }</pre> <p><em>[Note: The Index values in the SourceRow collection above relate to the row position in the Schema Map]</em>&nbsp;</p> <p>Now after the Synchronisation the SQL Table is populated from Podio and the Local Disk contains copies of the Files that were attached to the Form Files.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f4bb922a-6da6-4553-852c-021fcf52a6ed"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_2d3c6035-99fb-4b55-a235-9ee403738a9f" width="644" height="216"></a> </p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_2c0e30eb-37bd-4ea6-a068-b71c450e495b"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_ae726708-5496-4319-83da-46942dd43a81" width="644" height="331"></a> </p> <p>&nbsp;</p> <p>This is just a simple example of how Data Sync can integrate with Podio to provide business value quickly and easily.</p> Team Blog Thu, 14 Mar 2013 12:12:02 +0000 http://www.simego.com/Blog/2013/03/Connecting-to-Podio info@simego.com (Sean Cleaver) Connecting to Podio <p>We currently have two providers for Podio one for Space Contact Items and one for Space Apps.</p> <p>To connect to Podio you will need to locate the Podio providers in the list of Providers and the easiest way is to use the search function.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_8206d78b-afb3-495e-a2cd-70cba4859b7d"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_f6ea5c8d-afb7-4f2d-92c1-a0970aee4486" width="644" height="356"></a> </p> <p>&nbsp;</p> <p><em>[Note: Leaving the OAuth settings blank will allow Data Sync to use our default application settings with Podio, you can override these with your own Podio API keys if you like.]</em></p> <p>&nbsp;</p> <p>Click on the Credentials property and this will then bring up a web page where you need to authenticate with Podio and grant Data Sync permission to connect to your account.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d8d017fa-0e0f-48cf-8967-c4d167f5b6fa"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_2b39f735-1c7d-4004-8ea8-a3f313e622ae" width="644" height="354"></a> </p> <p>&nbsp;</p> <p>Once connected you can then pick the App in Podio to connect to from the <strong>App</strong> Drop-down list.</p> <p>&nbsp;</p> <p>The <strong>Silent</strong> property prevents updates to the app INSERT/UPDATE from being reported to your Activity Stream.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_2e77bbd4-0309-4abe-81d0-4a7ced58f6b6"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_cd74ed3d-58b4-4edf-8302-ca2632f91da8" width="644" height="356"></a> </p> <p></p> <p>&nbsp;</p> <p>Clicking OK will then popup the Connection Library option which is a new feature that allows you to save credentials for use again later. The OAuth credentials will expire after 28 days so it’s a good idea to save these in the connection library so you can easily update them again later.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f90f2310-2064-41fa-8b80-87e50502bdb6"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_6b556ed0-7658-490e-9a6c-07b1ca0c700c" width="524" height="261"></a> </p> <p>&nbsp;</p> <p>That’s it now your connected to Podio and you can start designing your integration or migration.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_05736be4-f2a4-4e72-aae3-3fe76311e07a"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_5c0fe8ae-bd66-40a5-95a7-934130ca4d62" width="644" height="401"></a></p> Team Blog Thu, 14 Mar 2013 11:26:07 +0000 http://www.simego.com/Blog/2013/02/Integration-with-Podio info@simego.com (Sean Cleaver) Integration with Podio <p>Here at Simego we love <a href="https://company.podio.com">Podio</a> so it was natural that we developed a connector for Podio that would allow us to integrate Data from our LOB systems into Podio.</p> <p>&nbsp;</p> <p>Out of the Box Podio lets you import Excel sheets and a few other options. But we need something that we can run on a Schedule which can take information captured through our Website and ensure it ends up in Podio.</p> <p>&nbsp;</p> <p>We&rsquo;ve made this all a bit too easy, simply connect Data Sync to your Source data and use the new Podio data provider. If your working with Contacts then you should use the Contacts provider. Set the Credentials which goes through the OAuth song and dance and then choose the Space in Podio to connect.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ceb46b47-9705-44f0-a687-91af94a01008"><img height="356" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_f9a3b0fe-04e4-4d08-8793-5072cbd566d4" alt="image" border="0" style="display: inline; border-width: 0px;" title="image" /></a></p> <p>&nbsp;</p> <p>Then Map your data just as you would any other system, you need a Unique Key to relate records and Podio exposes a nice <em>external_id</em> column you can use to store the ID from your source system.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_377db629-661f-4f05-a170-6cd07ea3b9b4"><img height="484" width="630" src="https://simegoimages.blob.core.windows.net/images/blog_1f728f96-ec8c-43ee-8b47-73123635402e" alt="image" border="0" style="display: inline; border-width: 0px;" title="image" /></a></p> <p>&nbsp;</p> <p>Then run through the normal Compare and Sync process and your new data will then be synchronised with data in Podio.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_859f7637-3597-47b5-975f-e2de7f66df4e"><img height="484" width="630" src="https://simegoimages.blob.core.windows.net/images/blog_9dbac137-e9e0-4253-baa6-802d53312ad5" alt="image" border="0" style="display: inline; border-width: 0px;" title="image" /></a></p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_90b00da5-1a00-4951-9802-a193e10bed5e"><img height="360" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_92e9a7de-25d8-4ae1-b45a-b8c683d788aa" alt="image" border="0" style="display: inline; border-width: 0px;" title="image" /></a></p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_7ba21db8-00f8-488e-bcf6-3bc3edf40766"><img height="484" width="577" src="https://simegoimages.blob.core.windows.net/images/blog_918e6be9-cc4a-46c2-9866-b1fa5443e1d7" alt="image" border="0" style="display: inline; border-width: 0px;" title="image" /></a></p> Team Blog Fri, 22 Feb 2013 14:33:21 +0000 http://www.simego.com/Blog/2013/02/EMC-Documentum-Integration david@simego.com (David Evans) EMC Documentum Integration <p>We have recently been working with some of our partners to allow them to integrate their client&rsquo;s &lsquo;Documentum&rsquo; environments using Data Sync Studio. Currently, this is still in development but early signs are that we will offer the integration as a product.</p> <p>We have developed the product to be purely api based and it needs no connections to the underlying Database or content servers.</p> <p>I am going to run through moving files between the desktop and a Documentum environment. We could just as easily move files from a Sharepoint Library to Documentum as well.</p> <p>As usual we use the following simple integration steps in DS3:</p> <ol> <li>Connect to a source which in this case is a folder on my desktop </li> <li>Connect to the destination which is a Documentum folder within a Repository </li> <li>Map any fields we require. This mapping is a feature built into DS3 and allows us to lookup information to update Documentum metadata about the document entity. </li> <li>Press the &lsquo;Synchronise&rsquo; button and the documents are sent up to Documentum with the meta data correctly set. </li> </ol> <p>For this demonstration we are going to use some simple text files as examples but we can target more complex types in Documentum.</p> <p>Step 1: Connecting to the file system source:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_1d308f99-e2e4-4897-bf9c-648e63851222"><img height="362" width="660" src="https://simegoimages.blob.core.windows.net/images/blog_76e43b13-3c8e-4f7c-a39e-61a3e61b6767" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>Select the folder containing the files you would like in Documentum.</p> <p>Step 2 : Connecting to a Documentum Repository and Selecting in Folder</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_59f31120-ff4e-443e-ab17-981659d82363"><img height="408" width="660" src="https://simegoimages.blob.core.windows.net/images/blog_d9137488-dc93-45e2-9406-f186d3f5d630" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>We have several options:</p> <p><strong>Credentials</strong>: Username, password and optionally Domain</p> <p><strong>DocumentType</strong>: The connector displays a dropdown listing all types within the repository.</p> <p><strong>DocumentumServerURL</strong>: The http address of the Documentum DFC url.</p> <p>Timeout: A timeout in of any request.</p> <p><strong>ContentUpdateBehaviour</strong>: Under some circumstances we are not interested in integrating the content of a document just the attributes (contentless documents). We can choose to synchronise the content or not.</p> <p><strong>Folder</strong>: The connector allows you to browse the folders within the Repository to find a folder to sync.</p> <p><strong>RepositoryName</strong>: The Repository Name.</p> <p>Step 3: Map the columns between the 2 Data Sources</p> <p>There are several things to note about the connector:</p> <ol> <li>It will automatically bring back the attributes for the selected &lsquo;type&rsquo; In this case we are just using the vanilla &lsquo;dm_document&rsquo; type in Documentum. </li> <li>It will use the &lsquo;dm_formats&rsquo; table to map files to their correct &lsquo;Format&rsquo; in Documentum. </li> <li>It supports repeating attributes with a string[], int[] attribute type. </li> <li>Dynamic Columns are fully supported if we need to lookup values to write to attributes in the target types. </li> </ol> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_a6ef035e-49da-458a-941a-df2e781c8465"><img height="398" width="660" src="https://simegoimages.blob.core.windows.net/images/blog_bd6b01b5-61c6-4ec8-b163-cd5ba8da3598" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>I added a new Dynamic Field called &lsquo;Subject&rsquo; to map to the dm_document attribute Subject:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_58e68dbd-bae4-47d8-8038-3417ccc196af"><img height="430" width="660" src="https://simegoimages.blob.core.windows.net/images/blog_3c5c606e-718b-473e-bd04-3c53ba6f898d" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>Step 3: Run the Synchronisation:</p> <p>Comparing A-&gt;B (File Systems &ndash;&gt; Documentum) and then synchronising moves the documents and updates the metadata accordingly:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_2d1b403c-cea5-450b-9563-2328c9a4fcb2"><img height="490" width="660" src="https://simegoimages.blob.core.windows.net/images/blog_91d72e3f-4623-4067-bd8d-62834bb07e04" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>When I delete some files from the source file system and change the Subject in the Dynamic Column to be &lsquo;Food and Beverage&rsquo; we can update the new files.</p> <p>Comparing Comparing A-&gt;B (File Systems &ndash;&gt; Documentum) again DS3 shows us the updates it will make to Documentum:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_be80455e-b348-4761-8f6f-dca6c7f36abb"><img height="384" width="660" src="https://simegoimages.blob.core.windows.net/images/blog_149a2478-7bc6-49dd-80b5-4d04fcdb3e32" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> <p>You might notice the performance times, there is no caching and the Documentum Server (Data Source B) is 3000 miles away over the internet.</p> <p>We can then see the update to the Subject in the Document properties:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_6f97d7c0-f574-43b5-8375-076543c57742"><img height="493" width="660" src="https://simegoimages.blob.core.windows.net/images/blog_7f102a64-855b-4545-a288-913691ab3a77" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>As we progress through the development of this connector I will post some more blogs.</p> <p>If you would like more information about this Documentum connector please just drop an email to <a href="mailto:sales@simego.com">sales@simego.com</a> and we can organise a chat over tea.</p> Team Blog Thu, 21 Feb 2013 15:45:30 +0000 http://www.simego.com/Blog/2013/01/Install-Data-Sync-Team-Server info@simego.com (Sean Cleaver) Install Data Sync Team Server <h3>System Requirements</h3> <p>&nbsp;</p> <ul> <li>Microsoft Windows Server 2008/R2/2012 <li>IIS 7.0 or Higher <li>Windows Authentication for IIS <li>.NET Framework 4.0 Full Profile <li>SQL Server 2008 or Higher configured for SQL Mixed Authentication Mode.</li></ul> <p>&nbsp;</p> <h3>Step 1 – Data Synchronisation Studio</h3> <p>Install <a href="http://www.simego.com/Install/Data-Synchronisation-Studio">Data Synchronisation Studio</a> as normal selecting either 64 bit or 32 bit as required for your system. Then activate Data Sync with your Server Edition License Key.</p> <h3>Step 2 – Data Sync Team Server</h3> <p>Install <a href="http://www.simego.com/Install/Ouvvi">Data Sync Team Server</a> as normal by running the install program this will install the IIS WebSite for Ouvvi and install the Ouvvi Processing Service. </p> <h3>Step 3 – DB Setup</h3> <p>The next step is to create an Ouvvi Database that this instance of Ouvvi will connect to. When you start the Ouvvi Website if there’s no configuration file it will start the setup process.&nbsp; Here you need to enter your Network Credentials these are your Windows Username &amp; Password that has Admin rights for your SQL Server. The SQL Server Name and Instance Name such as <em>LOCALHOST\SQLEXPRESS</em> the name of the Database you wish to create and a SQL User account name and password (Setup will create this user).</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f9bd5c25-9b61-464c-981e-fb655dadcee7"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_ee4c7b8b-9da4-45eb-a753-a7464655815a" width="593" height="484"></a></p> <h3>Step 4 – License</h3> <p>Once the DB is configured which takes a few seconds you will be redirected to the home page with further setup instructions. The first thing is to enter your Server Edition license key. </p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_1c12acb0-96e7-43bb-af6d-806ef1a580d4"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_487d0600-7737-4ac0-b1b0-c20eccfe77a5" width="644" height="369"></a> </p> <p>Selecting “Projects” from the Menu will re-direct you to the License Page. Enter your Key and Click Activate.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_a334a5da-23c6-4b74-b8e3-f44fb1b97658"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_719110ee-cef8-4bbd-9141-c0eec51f6956" width="644" height="297"></a> </p> <h3>Step 5 – Windows Service</h3> <p>Next you need to open the Windows Service Control Manager (services.msc) to configure and Start the Ouvvi Processing Service.</p> <p>Locate the “Ouvvi Processing Service” set the user to a domain service account you want to use to run the Data Sync integration Tasks and press Start.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_4ab8a142-65e4-4db7-bc29-e7ae84bd5295"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_dab3bb98-41b7-47ff-b77b-ae67a81dc5ce" width="644" height="319"></a> </p> <p>Now if you switch to the Services –&gt; System Services Tab you will see all the background Tasks have started and are running.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_26cf278e-bf8a-40f2-a893-d03a9347201c"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_9a55c63c-d219-4983-abca-834f68aaf1d2" width="644" height="270"></a> </p> <p>&nbsp;</p> <p>NOTE: If you need to re-start the setup process you should delete the Ouvvi.config file from the Directory C:\ProgramData\Simego\Ouvvi and restart the OuvviWeb AppPool in IIS.</p> Team Blog Mon, 07 Jan 2013 12:37:58 +0000 http://www.simego.com/Blog/2012/12/SQL-Change-Trigger-DS3-Team-Server-Feature info@simego.com (Sean Cleaver) SQL Change Trigger - DS3 Team Server Feature <p>Yesterday we blogged about the <a href="http://www.simego.com/Blog/2012/12/Using-Ouvvi-to-Enable-Real-Time-CRM-Synchronisation">CRM Trigger</a> that enables real-time synchronisation with <a href="http://www.simego.com/Products/Ouvvi-Console">Ouvvi / Data Sync Team Server</a>. Today we&rsquo;re going to cover off the other side where you can trigger a project to start from a change in your SQL Database (SQL Server, Oracle, DB2, MySQL, ODBC, OleDB etc).</p> <p>&nbsp;</p> <p>This works in a similar way to the CRM Trigger except this time we query your SQL Database every 30 seconds and ask for the latest <strong>DateTime</strong> value of any changes.</p> <p>&nbsp;</p> <p>To do this you need to write a query that would return the latest change timestamp from your database for the items your monitoring.</p> <p>&nbsp;</p> <h3>For example</h3> <p><strong>SELECT MAX(modifiedon) FROM dbo.AccountBase</strong></p> <p>To set this up you Add a new Trigger to your Data Sync Team Server environment.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_1ba4ebbf-3c9e-43e9-bcf7-f3c48520c1dc"><img height="92" width="507" src="https://simegoimages.blob.core.windows.net/images/blog_862a6584-3f4d-4d35-b6a0-73ec610dfe1e" alt="Add Trigger" style="display: inline; border-width: 0px;" title="Add Trigger" /></a></p> <p>Choose the SQL Trigger Option</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ff0fbc1c-86c8-40c4-9234-d7ed8796b3cc"><img height="266" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_649564d7-86e9-40b2-9cd6-874ac2070b5f" alt="New Trigger" style="display: inline; border-width: 0px;" title="New Trigger" /></a></p> <p>&nbsp;</p> <p>Then configure the Trigger, choosing the Network Library and connection string to your database and the Query to execute. (Yes you can use other database servers i.e. Oracle, IBM DB2, etc).</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_17f56d70-2d50-43df-b8cf-aa10d04f82e1"><img height="474" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_dc8580fc-30f3-4f79-aaea-cb0824a81ce5" alt="Edit Trigger" style="display: inline; border-width: 0px;" title="Edit Trigger" /></a></p> <p>&nbsp;</p> <p>That&rsquo;s it Ouvvi will start to Query your SQL DB every 30 seconds and when there&rsquo;s a change projects linked to this trigger will be queued to start.</p> <p>&nbsp;</p> <p>Now that the project is using a Trigger you can see under Reporting how the project is started through distribution of actual changes rather than a schedule.</p> <p>&nbsp;</p> <p>&nbsp;<a href="https://simegoimages.blob.core.windows.net/images/blog_ab7ae4c9-d1ef-4c55-bcd8-3889564da105"><img height="440" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_19ce55d0-2e62-4cd4-973d-5fcbe9f30ae9" alt="Reporting" border="0" style="display: inline; border: 0px;" title="Reporting" /></a></p> <p>&nbsp;</p> <p>If your Table does not have a valid DateTime column that gets updated when the record is changed then you can implement a solution based on SQL Triggers to maintain an updated timestamp value like this.</p> <p>&nbsp;</p> <p>Create a new SQL Table like this in your Database</p> <div style="overflow: auto; cursor: text; font-size: 8pt; font-family: 'Courier New', courier, monospace; direction: ltr; text-align: left; margin: 20px 0px 10px; line-height: 12pt; max-height: 200px; width: 97.5%; background-color: #f4f4f4; border: silver 1px solid; padding: 4px;" id="codeSnippetWrapper"> <div style="overflow: visible; font-size: 8pt; font-family: 'Courier New', courier, monospace; color: black; direction: ltr; text-align: left; line-height: 12pt; width: 100%; background-color: #f4f4f4; border-style: none; padding: 0px;" id="codeSnippet"> <pre style="overflow: visible; font-size: 8pt; font-family: 'Courier New', courier, monospace; color: black; direction: ltr; text-align: left; margin: 0em; line-height: 12pt; width: 100%; background-color: white; border-style: none; padding: 0px;"><span style="color: #606060;" id="lnum1"> 1:</span> <span style="color: #0000ff;">CREATE</span> <span style="color: #0000ff;">TABLE</span> [dbo].[OuvviTriggers] (</pre> <!--CRLF--> <pre style="overflow: visible; font-size: 8pt; font-family: 'Courier New', courier, monospace; color: black; direction: ltr; text-align: left; margin: 0em; line-height: 12pt; width: 100%; background-color: #f4f4f4; border-style: none; padding: 0px;"><span style="color: #606060;" id="lnum2"> 2:</span> [ID] <span style="color: #0000ff;">int</span> <span style="color: #0000ff;">IDENTITY</span>(1, 1) <span style="color: #0000ff;">NOT</span> <span style="color: #0000ff;">NULL</span>,</pre> <!--CRLF--> <pre style="overflow: visible; font-size: 8pt; font-family: 'Courier New', courier, monospace; color: black; direction: ltr; text-align: left; margin: 0em; line-height: 12pt; width: 100%; background-color: white; border-style: none; padding: 0px;"><span style="color: #606060;" id="lnum3"> 3:</span> [Name] nvarchar(255) <span style="color: #0000ff;">NOT</span> <span style="color: #0000ff;">NULL</span>,</pre> <!--CRLF--> <pre style="overflow: visible; font-size: 8pt; font-family: 'Courier New', courier, monospace; color: black; direction: ltr; text-align: left; margin: 0em; line-height: 12pt; width: 100%; background-color: #f4f4f4; border-style: none; padding: 0px;"><span style="color: #606060;" id="lnum4"> 4:</span> [Updated] datetime <span style="color: #0000ff;">DEFAULT</span>(getdate()) <span style="color: #0000ff;">NOT</span> <span style="color: #0000ff;">NULL</span></pre> <!--CRLF--> <pre style="overflow: visible; font-size: 8pt; font-family: 'Courier New', courier, monospace; color: black; direction: ltr; text-align: left; margin: 0em; line-height: 12pt; width: 100%; background-color: white; border-style: none; padding: 0px;"><span style="color: #606060;" id="lnum5"> 5:</span> )</pre> <!--CRLF--> <pre style="overflow: visible; font-size: 8pt; font-family: 'Courier New', courier, monospace; color: black; direction: ltr; text-align: left; margin: 0em; line-height: 12pt; width: 100%; background-color: #f4f4f4; border-style: none; padding: 0px;"><span style="color: #606060;" id="lnum6"> 6:</span> GO</pre> <!--CRLF--></div> </div> <p>&nbsp;</p> <p>Then you can create a SQL Trigger on any Table that you want to monitor like this where you update the &lsquo;TableA&rsquo; to match the name of the Table your monitoring.</p> <p>&nbsp;</p> <div style="overflow: auto; cursor: text; font-size: 8pt; font-family: 'Courier New', courier, monospace; direction: ltr; text-align: left; margin: 20px 0px 10px; line-height: 12pt; max-height: 200px; width: 97.5%; background-color: #f4f4f4; border: silver 1px solid; padding: 4px;" id="codeSnippetWrapper"> <div style="overflow: visible; font-size: 8pt; font-family: 'Courier New', courier, monospace; color: black; direction: ltr; text-align: left; line-height: 12pt; width: 100%; background-color: #f4f4f4; border-style: none; padding: 0px;" id="codeSnippet"> <pre style="overflow: visible; font-size: 8pt; font-family: 'Courier New', courier, monospace; color: black; direction: ltr; text-align: left; margin: 0em; line-height: 12pt; width: 100%; background-color: white; border-style: none; padding: 0px;"><span style="color: #606060;" id="lnum1"> 1:</span> <span style="color: #0000ff;">CREATE</span> <span style="color: #0000ff;">TRIGGER</span> TableA_OuvviTrigger</pre> <!--CRLF--> <pre style="overflow: visible; font-size: 8pt; font-family: 'Courier New', courier, monospace; color: black; direction: ltr; text-align: left; margin: 0em; line-height: 12pt; width: 100%; background-color: #f4f4f4; border-style: none; padding: 0px;"><span style="color: #606060;" id="lnum2"> 2:</span> <span style="color: #0000ff;">ON</span> [TableA]</pre> <!--CRLF--> <pre style="overflow: visible; font-size: 8pt; font-family: 'Courier New', courier, monospace; color: black; direction: ltr; text-align: left; margin: 0em; line-height: 12pt; width: 100%; background-color: white; border-style: none; padding: 0px;"><span style="color: #606060;" id="lnum3"> 3:</span> <span style="color: #0000ff;">FOR</span> INSERT, <span style="color: #0000ff;">UPDATE</span>, <span style="color: #0000ff;">DELETE</span></pre> <!--CRLF--> <pre style="overflow: visible; font-size: 8pt; font-family: 'Courier New', courier, monospace; color: black; direction: ltr; text-align: left; margin: 0em; line-height: 12pt; width: 100%; background-color: #f4f4f4; border-style: none; padding: 0px;"><span style="color: #606060;" id="lnum4"> 4:</span> <span style="color: #0000ff;">AS</span></pre> <!--CRLF--> <pre style="overflow: visible; font-size: 8pt; font-family: 'Courier New', courier, monospace; color: black; direction: ltr; text-align: left; margin: 0em; line-height: 12pt; width: 100%; background-color: white; border-style: none; padding: 0px;"><span style="color: #606060;" id="lnum5"> 5:</span> <span style="color: #0000ff;">UPDATE</span> [dbo].[OuvviTriggers]</pre> <!--CRLF--> <pre style="overflow: visible; font-size: 8pt; font-family: 'Courier New', courier, monospace; color: black; direction: ltr; text-align: left; margin: 0em; line-height: 12pt; width: 100%; background-color: #f4f4f4; border-style: none; padding: 0px;"><span style="color: #606060;" id="lnum6"> 6:</span> <span style="color: #0000ff;">SET</span> [Updated] = GETDATE()</pre> <!--CRLF--> <pre style="overflow: visible; font-size: 8pt; font-family: 'Courier New', courier, monospace; color: black; direction: ltr; text-align: left; margin: 0em; line-height: 12pt; width: 100%; background-color: white; border-style: none; padding: 0px;"><span style="color: #606060;" id="lnum7"> 7:</span> <span style="color: #0000ff;">WHERE</span> [Name] = <span style="color: #006080;">'TableA'</span></pre> <!--CRLF--> <pre style="overflow: visible; font-size: 8pt; font-family: 'Courier New', courier, monospace; color: black; direction: ltr; text-align: left; margin: 0em; line-height: 12pt; width: 100%; background-color: #f4f4f4; border-style: none; padding: 0px;"><span style="color: #606060;" id="lnum8"> 8:</span> GO</pre> <!--CRLF--></div> </div> <p>&nbsp;</p> <p>Then your Ouvvi Trigger just becomes <strong>SELECT [Updated] FROM [OuvviTriggers] WHERE [Name] = &lsquo;TableA&rsquo;</strong></p> <p><strong><br /></strong></p> Team Blog Tue, 11 Dec 2012 11:05:15 +0000 http://www.simego.com/Blog/2012/12/Using-Ouvvi-to-Enable-Real-Time-CRM-Synchronisation info@simego.com (Sean Cleaver) CRM 2011 Real Time Synchronisation - DS3 Team Server Feature <p>DS3 Team Server or&nbsp;<a href="http://www.simego.com/Products/Ouvvi-Console">Ouvvi</a> has a Triggering system that can be used to provide almost real-time synchronisation. Internally <a href="http://www.simego.com/Products/Ouvvi-Console">Ouvvi</a> has a 30 second clock that evaluates configured Triggers and when they need to be fired any projects associated with that trigger will be queued for execution. This feature is completely independent of your CRM system and does not require any special installation onto your CRM Server.</p> <p>&nbsp;</p> <p>One Trigger that Ouvvi has is the <strong>CRM Entity Trigger,</strong> this trigger queries an Entity in CRM for any updates since the last change (createdon, modifiedon). If this changes then the Trigger is fired. The benefit to this method is that changes are replicated quickly and unnecessary scheduled syncs do not need to occur.</p> <p>To create a new <strong>CRM Entity</strong> Trigger go to the Triggers view and choose Add Trigger</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_fe340336-4836-4ab2-bc1b-f58f235c1626"><img height="126" width="250" src="https://simegoimages.blob.core.windows.net/images/blog_8a953a34-f005-4fe2-8a79-28260da28af2" alt="Add Trigger" border="0" style="display: inline; border: 0px;" title="Add Trigger" /></a></p> <p>Then choose the <strong>CRM Entity</strong> Trigger</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_5075c66d-9c6a-432b-b79e-90f0a307ad0f"><img height="388" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_2fefc5d1-c349-4d43-bd9c-a5da24c7b11e" alt="Select Trigger" border="0" style="display: inline; border: 0px;" title="Select Trigger" /></a></p> <p>Then configure the Trigger by entering your CRM Server Details and the Entity you want to monitor. (leave credentials blank to use the Ouvvi service credentials)</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_23e85944-9e08-4245-83c8-c9076d24884d"><img height="484" width="605" src="https://simegoimages.blob.core.windows.net/images/blog_4abbcd27-bc8a-456c-9334-515e56fcd76d" alt="Configure Trigger" border="0" style="display: inline; border: 0px;" title="Configure Trigger" /></a></p> <p>That&rsquo;s it &hellip;. you simply then attach this trigger to the Ouvvi projects you want to execute when changes are made to this entity. It&rsquo;s important that your projects are configured to run in the Single Instance mode so that they do not run in Parallel.</p> Team Blog Mon, 10 Dec 2012 10:28:04 +0000 http://www.simego.com/Blog/2012/12/Connect-to-SharePoint-2013 info@simego.com (Sean Cleaver) Connect to SharePoint 2013 <p>For those of you planning to upgrade to SharePoint 2013 we thought we would let you know that DS works just fine with SharePoint 2013.</p> <p>Connect as normal – we recommend you start using the <strong>Client API</strong> Provider for new SharePoint projects.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_2a1f2299-4a44-40de-bdda-b3fd27d05107"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_2daf6791-f23d-431f-9c9b-14ec85797596" width="644" height="356"></a> </p> <p>Browse your Site Schema to find the List you want to connect to.</p> <p></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_e616e39a-a6e0-459a-bf00-9a4f539f7ede"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_fd0fe31f-ecfc-4023-bb85-3f9cd6503dcc" width="500" height="502"></a> </p> <p>Map your source to your target as normal</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d741757a-f515-4e0d-9692-ea22c10b3882"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_19189fc1-7767-4b5d-9055-76d19ed399b4" width="644" height="438"></a> </p> <p>Once you synchronise your Data is now presented in SharePoint 2013.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_6a93759d-6df6-4e8c-917d-d3a1c7fbc723"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_9135e517-d474-40e8-84c7-89bf125a5e7b" width="642" height="484"></a></p> Team Blog Tue, 04 Dec 2012 16:49:13 +0000 http://www.simego.com/Blog/2012/12/Handling-SharePoint-Lookup-Columns info@simego.com (Sean Cleaver) Handling SharePoint Lookup Columns <p>The <strong>SharePoint 2010 (Client API)</strong> Provider has an inbuilt feature to automatically match and link Lookup Columns based on the value associated with the List. </p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_fdf3cec0-61a7-4026-94d8-8460bcbd4895"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_57b1bfc1-42c5-4f4a-bb74-576f6ea466a1" width="644" height="356"></a> </p> <p>In this example we have created an Orders list and added one column that is a multi-value column that will contain the “Products” that relate to this Order.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_73701d46-0725-47dc-b536-4d71fd7e05b7"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_dc27ef0f-c643-4913-a2a7-0accce91f22f" width="231" height="484"></a> </p> <p>To handle the lookup data we simply need to provide the name of the lookup value separated with a semi-colon(;) i.e. here we created an Excel file with the order details.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_c0021752-8893-48e1-b5d2-ff48fc2262ea"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_0f24eb97-cd28-4b0f-ad84-a04ca8735897" width="644" height="418"></a> </p> <p>We then simply create a DS Project to Map this to our Orders List</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_8af81ec9-e316-4dea-bcb1-1fe9baccbe5a"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_e2e86a3e-d4b3-4f1e-88da-08e31a454d17" width="644" height="364"></a> </p> <p>Then once we synchronise this data the lookup is automatically connected.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_604427ad-feac-4ede-abad-d2792143a05e"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_a6253998-07bf-4be9-a6c6-1efe44d9e2f5" width="644" height="179"></a> </p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_b783c723-1238-4288-ac9f-4f7bfc7936c5"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_ad8956bd-83a0-4a09-9e2b-f1ed584061ec" width="644" height="466"></a> </p> <p>During the Synchronisation we download the list items from the related SharePoint List and index into this list to get SharePoint’s internal identifier number to assign to the item being linked. Therefore items must be an exact match and they must exist for the link to be successfully created.</p> <p>If your Lookup is simply a single item then you just need to return the Name/Title of that item.</p> Team Blog Mon, 03 Dec 2012 15:29:59 +0000 http://www.simego.com/Blog/2012/11/Connect-to-Dynamics-CRM-2011-Online info@simego.com (Sean Cleaver) Connect to Dynamics CRM 2011 Online <p>To Connect to Dynamics CRM 2011 Online with Data Synchronisation Studio you will need the URL to your CRM Environment.</p> <p>For example one test environment of ours is at https://simegoltd.crm4.dynamics.com for data sync you need to remove the Organisation name part so this becomes https://crm4.dynamics.com</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_cc56b2d5-6888-4594-a25d-2ab74eb676a3"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_0afd494a-4867-4984-9fb1-2a4aa74c276e" width="319" height="87"></a> </p> <p>This environment is pre-Office 365 Integration therefore if you have CRM Linked to your Office 365 Environment on one of the E plans then the Url will most likely be <a href="https://crm.dynamics.com">https://crm.dynamics.com</a></p> <p>You then Enter this into the <strong>CrmServerUrl</strong> property of the Provider, For CRM Online with LiveId choose the LiveId Authentication Provider, for Office 365 Choose Federated.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_9a97c707-cb05-4f40-b49e-a389028fe5f7"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_f3dc72ff-b09f-4581-abd7-203ac75378e3" width="644" height="356"></a> </p> <p>Next Enter your credentials under Authentication and then Choose your Organisation name from the Drop-down this might take a little while as it discovers the Organisations that are linked to your account.</p> <p></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_279e14d0-31ad-47c2-a7ba-25a8fa418587"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_907d1cc7-fdbd-4ff7-9105-65f8d7f1319a" width="644" height="356"></a> </p> <p>Next Choose an Entity to work with from the Entity Drop-down.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ce4ec5ae-57ee-4116-b6d7-392ffe2d8abc"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_34901af8-55a5-4728-8149-2dff743c86c3" width="644" height="356"></a> </p> <p>Then Click OK to connect.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_cad73a05-8557-4cbd-9111-7f658ee318cd"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_c790f1ff-8c48-408a-9e99-350635fc8b0c" width="644" height="311"></a></p> <p>&nbsp;</p> <h3>Issues</h3> <p>If you get an error message when you try to connect that the system cannot load the Microsoft.IdentityModel assembly&nbsp; then you need to install the <a href="http://www.microsoft.com/en-us/download/details.aspx?id=17331">Windows Identity Framework</a> (WIF) then re-start Data Sync for the changes to take affect.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_1e79a659-24fd-478d-8734-245c78a9bb3f"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_f111fec6-7a53-4077-81f5-2a9b2a476a3b" width="492" height="185"></a></p> Team Blog Fri, 30 Nov 2012 15:19:57 +0000 http://www.simego.com/Blog/2012/11/Getting-Starting-with-a-Trial-Evaluation david@simego.com (David Evans) Getting Starting with a Trial Evaluation <h3>Introduction - DS3 is so simple you can just 'Get Started'!</h3> <p>Data Synchronisation Studio v3 (DS3) is moves data between computer systems without the cost, effort and risk associated with developing a system integration solution. You point DS3 at the data source and then the target and DS3 makes one side equal to the other.</p> <p>In more advanced cases DS3 will make both sides the same! Make change on either side and it synchronises with the other.</p> <p>&nbsp;</p> <h3>Starting an Evaluation</h3> <p><strong>Step 1:</strong> Download the latest trial from <a href="http://www.simego.com/Install/Data-Synchronisation-Studio">http://www.simego.com/Install/Data-Synchronisation-Studio</a></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_5d1c218c-0a67-42c8-971a-ddc0cd55ec81"><img height="339" width="416" src="https://simegoimages.blob.core.windows.net/images/blog_326b73d2-4d3a-4601-a751-f556e075883d" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p><strong>&nbsp;</strong>&nbsp;</p> <p><strong>Step 2:</strong> Click the zip file and then click on the Data Sync installer</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_aa063af8-9f81-46b0-9049-db6b8c1a9e57"><img height="156" width="593" src="https://simegoimages.blob.core.windows.net/images/blog_af52d47d-ac72-4b19-9c63-53747da276fe" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p><strong>&nbsp;</strong>&nbsp;</p> <p><strong>Step 3:</strong> Approve running the installer</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_3282284f-0465-4331-8f81-9b3711901ef2"><img height="348" width="487" src="https://simegoimages.blob.core.windows.net/images/blog_40c4b6d6-2c32-488d-a3a6-528bebeb2b5c" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p><strong>&nbsp;</strong>&nbsp;</p> <p><strong>Step 4:</strong> Install the application and Finish</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_57a897f9-ae16-41e2-b774-4e4d433a2c65"><img height="328" width="420" src="https://simegoimages.blob.core.windows.net/images/blog_a09b4299-e6f6-4a81-84d8-7ac5203121c8" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p><strong>&nbsp;</strong>&nbsp;</p> <p><strong>Step 5:</strong>&nbsp; Run Data Sync for the first time:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_66ee1f33-fce4-4f1b-b39a-94c81664978c"><img height="439" width="367" src="https://simegoimages.blob.core.windows.net/images/blog_4eb1b2ab-8023-4377-900f-7e9d05ba2a06" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <h3>&nbsp;</h3> <h3>Getting a Trial License for DS3</h3> <p>&nbsp;</p> <p>Once started DS3 will ask for a key. Just send an Email to us requesting a Trial/Evaluation Key to support@simego.com, we will send you back a trial key within a few minutes and if you give us some detail on what you need to integrate eg, CRM + SQL, we have more quick start guides for many situations.</p> <p>&nbsp;</p> <p><strong>Step 1:</strong> Request Key.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ca97396c-9303-4ba5-8fed-08365fa51596"><img height="358" width="435" src="https://simegoimages.blob.core.windows.net/images/blog_6ba0cd47-253c-4569-a698-c9d424cead22" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>&nbsp;</p> <p><strong>Step 2:</strong> When you have your response back from support it will contain a key like this (this is not a valid Key!!):</p> <p><em><strong>QETSC3-SWSEZH-SRS4XM-S4EQHX-1YW4U6-HQ1BWA-U</strong></em></p> <p><strong><em>&nbsp;</em></strong>&nbsp;</p> <p><strong>Step 3:</strong> Click on the &lsquo;Enter License Key&rsquo; button and paste in the Key.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_8fbe5b9e-be04-49db-8417-71c2437c85c1"><img height="376" width="475" src="https://simegoimages.blob.core.windows.net/images/blog_a3585295-ac56-48c1-beed-37dc2d372e25" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>&nbsp;</p> <p><strong>Step 4:</strong> Click the OK Button to enter Data Synchronisation Studio</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ab515e34-709c-4dae-aef4-ddbc02b6f9d7"><img height="504" width="660" src="https://simegoimages.blob.core.windows.net/images/blog_28ccb7a5-6225-47fa-83ec-2494c140b632" alt="Screen Shot 2012-11-30 at 11.05.49" border="0" title="Screen Shot 2012-11-30 at 11.05.49" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> Team Blog Fri, 30 Nov 2012 09:41:55 +0000 http://www.simego.com/Blog/2012/11/Using-Lookups-to-Transform-Values info@simego.com (Sean Cleaver) Using Lookups to Transform Values <p>In this simple example we will demonstrate how you can use the Lookup functionality in Data Synchronisation Studio to Transform/Convert a value via an Lookup Data source.</p> <p>Our source data contains the numbers 1-10 and we want to convert these into the names of the Numbers.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_bd8f2451-c84d-41b8-a5d3-0dd9df8c2eef"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_d1c94170-cf70-4f17-b2a6-909a37b760a1" width="196" height="321"></a> </p> <p>We need then to create a Transformation Data source and for this we’re going to use Excel. We could use virtually anything we like, SQL, CRM, SharePoint, XML, CSV etc but Excel is ideal for this.</p> <p>This Excel file contains the From and To values for the Transformation, we need to create Column headers in the First row and then the Data below i.e.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_2991c991-f28b-460a-9859-3ad36862fae9"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_ce752eac-f93d-4468-bae5-b09af4552636" width="533" height="590"></a> </p> <p>Now we add the Lookup to the column (select the ID column, and press the ‘Add Lookup’ toolbar button) that we want to Transform, in our Example this is the ID column. First we select the Data source which for this example is Excel and browse to the Path of the Excel file. </p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_b0b2589c-f117-47ae-99f9-f3984b39e38c"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_4a940e73-3be2-47bc-b9b2-89eb5123d531" width="644" height="357"></a> </p> <p>Now we define the Lookup connection details, so we want to join the Lookup based on the Value in the <strong>From</strong> Column to the Value in the <strong>ID</strong> column in our source data. We also specify the Data Type as <strong>Int32</strong> since this is a Number 1-10.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_494a9c96-2ec8-4001-a803-dcf5f4c90023"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_3d3e17af-b657-4a7d-a683-9770d53df641" width="394" height="484"></a> </p> <p>We then add the “Lookup1_To” column from the Data source to our Schema Map so that we return the Value of the Lookup Transformation.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_be445ce6-d520-4015-a2b1-8bd3ea67bd40"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_93a5b889-400d-4413-a193-ae2f1d0cd023" width="644" height="169"></a> </p> <p>There we have it a Transformation from an External Data source via Lookups.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_77f69876-9d0f-4882-92f8-843392879b19"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_8833afc4-ce06-46e4-8cd3-99dd4314d014" width="312" height="327"></a> </p> <p>This simple Transformation could be created in code via Dynamic Columns but this shows a no-code solution and has the added benefit that you can easily extend the transformation values by updating the Excel data source.</p> Team Blog Tue, 27 Nov 2012 10:42:34 +0000 http://www.simego.com/Blog/2012/11/Connect-to-Office-365-Exchange-Server info@simego.com (Sean Cleaver) Connect to Office 365 Exchange Server <p>To connect to Exchange Server with Data Synchronisation Studio you need to get the URL for your Exchange Server. First go to your Outlook Web Access and the URL for your Exchange server.</p> <p>For example this one is https://dbxprd0710.outlook.com/owa/?exsvurl=1&amp;realm=office365.com&amp;wa=wsignin1.0</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d2003fa0-6ee2-4019-b383-3cf4be2f3d06"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_4896bacf-e6fc-4d08-9795-73c1113b2328" width="644" height="132"></a> </p> <p>This URL need to be modified so that it points to the Exchange Web Services URL so we take the server part and change it to https://dbxprd0710.outlook.com/ews/Exchange.asmx&nbsp;</p> <p>We then take this URL and enter it into the ExchangeUrl property of the “Exchange Web Services (Appointments)” provider.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_b4da8b81-efe4-4eb6-89d3-a573673c1411"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_5b4c9123-135e-45ff-8dd1-110ee20b3c97" width="644" height="356"></a> </p> <p>We then enter our Exchange Credentials</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ae112b11-078e-414a-a33f-51bd79222469"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_9e30f91a-1fbe-4cd5-8254-d348feb0e17b" width="644" height="356"></a> </p> <p>We then add our Mailbox name to the Mailboxes collection, this details which mailbox data we wish to return.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_da59bd22-71b0-4e72-bb80-456f938a80ee"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_1e58f249-872d-49dd-a719-656886bd4b2d" width="644" height="355"></a> </p> <p>And that’s it now you can return data from Office 365 Exchange Server.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_359329bd-f47c-437c-b4a3-119f77240e04"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_341d6745-0f28-417b-bbbd-160de44a33bd" width="644" height="368"></a></p> Team Blog Tue, 27 Nov 2012 10:17:26 +0000 http://www.simego.com/Blog/2012/11/Connect-to-365-SharePoint-Online info@simego.com (Sean Cleaver) Connect to 365 SharePoint Online <p>To connect to a List/Document Library with Data Synchronisation Studio on SharePoint Online you need to get the URL to your site.</p> <p>For example in the screenshot below our Team Site URL would be http://crmonline4u.sharepoint.com/TeamSite</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f145aa17-69ad-42a0-9bf6-a6c5084e0771"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_b10bc5b0-d02c-46ba-bcaa-c9d415681e91" width="575" height="207"></a> </p> <p>You then enter this URL into the <strong>SharePointUrl</strong> Property of the “Microsoft Office 365 SharePoint Online” Data Provider in Data Sync.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_07631713-da5f-49d2-b71e-3fffa5c7ca48"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_1b1f1888-ca67-42ae-a827-32d7275c0eb3" width="644" height="349"></a> </p> <p>You then need to enter the credentials for your SharePoint Online environment. Note that we do not currently support environments where you have federated authentication enabled to your own AD, standard 365 Authentication is fine.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_04e9fa8b-4b1d-4dc2-9976-3476b2980993"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_4bdb1f12-a21c-4ba7-b266-2bf38bee85c5" width="644" height="350"></a> </p> <p>Next select the List that you wish to connect to from the <strong>ListName</strong> drop down.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_fbc34a0b-d83d-4aac-af78-616ac5f263de"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_471011f3-73f3-41e9-8a7e-471bae269e22" width="644" height="352"></a> </p> <p>And click OK to connect and there you have it!</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_7eadc0ca-4e66-4f7c-9aac-1e493d6999e4"><img title="image" style="border-left-width: 0px; border-right-width: 0px; border-bottom-width: 0px; display: inline; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_4ff7a69c-15d9-4dd2-87aa-c916e0eeaeb6" width="644" height="206"></a></p> <p>&nbsp;</p> <h3>Issues</h3> <p>If you get an error message when you try to connect that the system cannot load the Microsoft.IdentityModel assembly&nbsp; then you need to install the <a href="http://www.microsoft.com/en-us/download/details.aspx?id=17331">Windows Identity Framework</a> (WIF) then re-start Data Sync for the changes to take affect.</p> <p>&nbsp;</p> <p>&nbsp;<img alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_f111fec6-7a53-4077-81f5-2a9b2a476a3b"></p> Team Blog Tue, 27 Nov 2012 10:04:02 +0000 http://www.simego.com/Blog/2012/11/Importing-CRM-2011-Connections info@simego.com (Sean Cleaver) Importing CRM 2011 Connections <p>To import/synchronise connection data in Dynamics CRM 2011 with Data Synchronisation Studio you need to define your schema map like this below.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_08375094-c66c-415a-bb60-7e701a4eaa91"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_43a40d26-9b2f-452f-87fe-f61a6509ddf3" width="644" height="236"></a> </p> <p><strong>record1idobjecttypecode</strong> and <strong>record2idobjecttypecode</strong> should return the names of the related entities for example.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_63304f06-ab81-4fc4-969c-185d1ed0a05a"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_4ef33224-0104-4a6c-9f0b-e65f4878c322" width="644" height="92"></a> </p> <p>CRM will automatically create the reverse connection which is a problem for Data Sync as it will see an extra record on the next sync. Therefore we filter this out by using an <strong>FetchXmlFilterExpression</strong> and just select the rows where <strong>ismaster</strong> is true as below.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_c2d49028-241a-4026-9e9b-3961780ee970"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_72c4846d-c173-42d0-9bff-c2187b62a0fa" width="194" height="244"></a> </p> <p>&nbsp;</p> <div id="codeSnippetWrapper" style="overflow: auto; cursor: text; font-size: 8pt; border-top: silver 1px solid; font-family: 'Courier New', courier, monospace; border-right: silver 1px solid; border-bottom: silver 1px solid; padding-bottom: 4px; direction: ltr; text-align: left; padding-top: 4px; padding-left: 4px; margin: 20px 0px 10px; border-left: silver 1px solid; line-height: 12pt; padding-right: 4px; max-height: 200px; width: 97.5%; background-color: #f4f4f4"> <div id="codeSnippet" style="border-top-style: none; overflow: visible; font-size: 8pt; border-left-style: none; font-family: 'Courier New', courier, monospace; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; line-height: 12pt; padding-right: 0px; width: 100%; background-color: #f4f4f4"><pre style="border-top-style: none; overflow: visible; font-size: 8pt; border-left-style: none; font-family: 'Courier New', courier, monospace; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; margin: 0em; line-height: 12pt; padding-right: 0px; width: 100%; background-color: white"><span id="lnum1" style="color: #606060"> 1:</span> <span style="color: #0000ff">&lt;</span><span style="color: #800000">filter</span> <span style="color: #ff0000">type</span><span style="color: #0000ff">="and"</span><span style="color: #0000ff">&gt;</span></pre><!--CRLF--><pre style="border-top-style: none; overflow: visible; font-size: 8pt; border-left-style: none; font-family: 'Courier New', courier, monospace; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; margin: 0em; line-height: 12pt; padding-right: 0px; width: 100%; background-color: #f4f4f4"><span id="lnum2" style="color: #606060"> 2:</span> <span style="color: #0000ff">&lt;</span><span style="color: #800000">condition</span> <span style="color: #ff0000">attribute</span><span style="color: #0000ff">="ismaster"</span> <span style="color: #ff0000">operator</span><span style="color: #0000ff">="eq"</span> <span style="color: #ff0000">value</span><span style="color: #0000ff">="true"</span> <span style="color: #0000ff">/&gt;</span></pre><!--CRLF--><pre style="border-top-style: none; overflow: visible; font-size: 8pt; border-left-style: none; font-family: 'Courier New', courier, monospace; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; margin: 0em; line-height: 12pt; padding-right: 0px; width: 100%; background-color: white"><span id="lnum3" style="color: #606060"> 3:</span> <span style="color: #0000ff">&lt;/</span><span style="color: #800000">filter</span><span style="color: #0000ff">&gt;</span></pre><!--CRLF--></div></div> Team Blog Mon, 26 Nov 2012 12:20:34 +0000 http://www.simego.com/Blog/2012/11/Empty-String-and-NULL-Behaviour info@simego.com (Sean Cleaver) Empty String and NULL Behaviour <p>By Default Data Sync detects NULL and an Empty String as a difference, however certain Data Sources such as CSV and SharePoint cannot differentiate between NULL and Empty String. Therefore we automatically override this behaviour for you when using these providers.</p> <p>If you want to force NULL and Empty String to be considered Equal during the comparison you can do this within the Project Automation feature. Simply set the Property <strong>EmptyStringAndNullAreEqual</strong> to True on one of the Data source Tables using the <strong>BeforeCompare</strong> Method.</p> <h3>For Example</h3> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_592c3bda-a801-40af-b980-40f4b6e1375a"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_6ac63005-02c8-44da-afcf-0ef88ebdad65" width="615" height="484"></a> </p> <p>&nbsp;</p> <div id="codeSnippetWrapper" style="overflow: auto; cursor: text; font-size: 8pt; border-top: silver 1px solid; font-family: 'Courier New', courier, monospace; border-right: silver 1px solid; border-bottom: silver 1px solid; padding-bottom: 4px; direction: ltr; text-align: left; padding-top: 4px; padding-left: 4px; margin: 20px 0px 10px; border-left: silver 1px solid; line-height: 12pt; padding-right: 4px; max-height: 200px; width: 97.5%; background-color: #f4f4f4"> <div id="codeSnippet" style="border-top-style: none; overflow: visible; font-size: 8pt; border-left-style: none; font-family: 'Courier New', courier, monospace; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; line-height: 12pt; padding-right: 0px; width: 100%; background-color: #f4f4f4"><pre style="border-top-style: none; overflow: visible; font-size: 8pt; border-left-style: none; font-family: 'Courier New', courier, monospace; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; margin: 0em; line-height: 12pt; padding-right: 0px; width: 100%; background-color: white"><span id="lnum1" style="color: #606060"> 1:</span> <span style="color: #0000ff">public</span> <span style="color: #0000ff">override</span> <span style="color: #0000ff">void</span> BeforeCompare(ProjectAutomationLoadResult loadResult)</pre><!--CRLF--><pre style="border-top-style: none; overflow: visible; font-size: 8pt; border-left-style: none; font-family: 'Courier New', courier, monospace; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; margin: 0em; line-height: 12pt; padding-right: 0px; width: 100%; background-color: #f4f4f4"><span id="lnum2" style="color: #606060"> 2:</span> {</pre><!--CRLF--><pre style="border-top-style: none; overflow: visible; font-size: 8pt; border-left-style: none; font-family: 'Courier New', courier, monospace; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; margin: 0em; line-height: 12pt; padding-right: 0px; width: 100%; background-color: white"><span id="lnum3" style="color: #606060"> 3:</span> loadResult.TableA.EmptyStringAndNullAreEqual = <span style="color: #0000ff">true</span>;</pre><!--CRLF--><pre style="border-top-style: none; overflow: visible; font-size: 8pt; border-left-style: none; font-family: 'Courier New', courier, monospace; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; margin: 0em; line-height: 12pt; padding-right: 0px; width: 100%; background-color: #f4f4f4"><span id="lnum4" style="color: #606060"> 4:</span> }</pre><!--CRLF--></div></div> Team Blog Mon, 26 Nov 2012 11:00:44 +0000 http://www.simego.com/Blog/2012/08/Ouvvi-Data-Sync-Team-Server-Configuration-Management info@simego.com (Sean Cleaver) Ouvvi Data Sync Team Server - Configuration Management <p>With Ouvvi Data Sync Team Server you can manage Data Sync project settings from a Central Location with the &lsquo;User Settings&rsquo; page in Ouvvi.</p> <p>This allows you to easily switch Databases from Test to Live, specify credentials etc.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_488ba6d0-0556-4447-bedb-c72525acdf30"><img height="474" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_df3e0cae-e961-472b-a722-e390df845ba9" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>&nbsp;</p> <p>When the Step is run inside Ouvvi or downloaded the configuration variables are expanded so that the value in the settings list is replaced with the value in the Data Sync project.</p> <p>&nbsp;</p> <p>To enable a setting on a project to be managed this way you must edit the Data Sync project file before uploading into Ouvvi.</p> <p>&nbsp;</p> <p>Open the project in Notepad and locate the configuration variables.</p> <p>For example to enable the SharePointUrl to be managed this way add a value in ConfigKey below.</p> <p>&nbsp;</p> <p><em>so this</em></p> <p>&lt;Parameter Name="SharePointUrl" Value="<a href="http://deathstar/test&quot;">http://deathstar/test"</a> ConfigKey="" /&gt;</p> <p>&nbsp;</p> <p><em>becomes this</em></p> <p>&lt;Parameter Name="SharePointUrl" Value="<a href="http://deathstar/test&quot;">http://deathstar/test"</a> ConfigKey="<strong>OurSharePointSite</strong>" /&gt;</p> <p>&nbsp;</p> <p>Then when you upload the project into Ouvvi it will create a new User Setting for <strong>OurSharePointSite</strong> with the current value that allows you to easily switch the value at runtime.</p> Team Blog Fri, 31 Aug 2012 17:32:35 +0000 http://www.simego.com/Blog/2012/08/Importing-Active-Directory-Contacts info@simego.com (Sean Cleaver) Importing Active Directory Contacts <p>This guide shows how you can Import/Export or Synchronise Contact Data with Active Directory with Data Synchronisation Studio. (Requires Data Synchronisation Studio Release 3.0.586 or greater)</p> <p>In this Example we have created a new OU called ‘Contacts’ that we will Synchronise our Contact Data into. </p> <p>&#160;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_b90ded6d-3714-4ea2-80e5-423e7d5b5787"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_ac53d0fb-1e88-49f6-ab5c-0950755ec2e6" width="575" height="336" /></a></p> <p>&#160;</p> <h3>Connect Data Source A</h3> <p>&#160;</p> <p>We have created some sample Data and connected it to Data Source A to Synchronise with Active Directory. In this example we use the UserCode column as our Unique Key and we will store this value in the EmployeeId attribute in Active Directory to maintain the link between records.</p> <p>&#160;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_aecd8c5c-c3d9-4894-a98f-cf82e9cf7462"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_4d108903-593d-471f-92d2-241c49682826" width="644" height="315" /></a></p> <p>&#160;</p> <h3>Connect Data Source B</h3> <p>&#160;</p> <p>We need to specify a correctly formed LDAP Connection such as the one below for this.</p> <p><strong>LDAP://OU=Contacts,DC=corp,DC=simego,DC=com</strong></p> <p>We also need to modify the <strong>LDAPFilter</strong> property to return objects of Type ‘Contact’ for example <strong>(&amp;(objectCategory=contact))</strong></p> <p><strong></strong></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_7c77d9d1-894d-4a8f-b5df-ed3553bcc0a0"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_3b811a5b-98bc-4b59-8015-7d49447c0c63" width="644" height="356" /></a></p> <p>&#160;</p> <p>Next we need to adjust a few properties on the Active Directory provider to enable the creation of new records in your AD.</p> <p>Set the following properties</p> <p>SchemaClassName to ‘Contact’</p> <p>SchemaItemFormat to ‘$FirstName$ $Surname$’</p> <p>EnableAdd to ‘True’</p> <p>EnableDelete to ‘True’ (Optional)</p> <p>&#160;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f43b4407-4046-4d67-aafa-5d0b6271a625"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_ca9058f4-0214-4c5c-b7ea-951bcc1bdd32" width="377" height="384" /></a></p> <p>&#160;</p> <p>Then Setup your Schema Map to Map your Source Columns to Active Directory and in this case using the EmployeeID to Store an Identifier to the Contact record.</p> <p>&#160;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_538c3b65-edf6-4592-8ea4-fd44ed39714e"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_5ebbad9d-cffb-47ad-9463-1569d03d93e4" width="644" height="193" /></a></p> <p>&#160;</p> <p>Now when you Compare and Synchronise the records are created in your AD.</p> <p>&#160;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_325c799a-a87b-4b37-8d9f-a96612dffd44"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_df1a9329-3739-4468-a5b0-6055cc55d5ec" width="644" height="308" /></a></p> <p>&#160;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_12a46272-7bfd-445a-9175-a38bad0c8304"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_69cd4c72-f465-4b7d-bf6a-e28b872f0f08" width="416" height="461" /></a></p> Team Blog Fri, 31 Aug 2012 14:35:22 +0000 http://www.simego.com/Blog/2012/08/Dynamics-CRM-2011-N-N-Relationships info@simego.com (Sean Cleaver) Dynamics CRM 2011 N:N Relationships <p>This guide will demonstrate how to create N:N (Many to Many) Relationships in CRM 2011 with Data Synchronisation Studio. (requires Data Sync release 3.0.586 or greater)</p> <p>For this example we will use out of the box entities and show how to relate 1 or more Competitors to Opportunities.</p> <p>&nbsp;</p> <p>Below shows an existing Opportunity linked to 3 Competitors in CRM we will create a Data Sync project that can create this automatically.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_57e465a2-e260-48fa-9f35-f12886670b39"><img height="484" width="611" src="https://simegoimages.blob.core.windows.net/images/blog_1cd2dc05-fe10-4358-abcd-0fe11dbc8457" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> <p>&nbsp;</p> <h3>Configure Source Data</h3> <p>&nbsp;</p> <p>First up we need to configure what it is were going to import/synchronise with CRM. To keep things simple we will create a CSV File containing the Target Relationships.</p> <p><strong>&nbsp;</strong></p> <p><strong>CSV Data</strong></p> <p>Opportunity,Competitor <br />Opportunity 1, Competitor 1 <br />Opportunity 1, Competitor 2 <br />Opportunity 1, Competitor 3 <br />Opportunity 2, Competitor 1 <br />Opportunity 2, Competitor 2</p> <p>&nbsp;</p> <p>This Data shows we are Mapping &lsquo;Opportunity 1&rsquo; to Competitor 1,2 and 3 and &lsquo;Opportunity 2&rsquo; to Competitor 1 and 2.</p> <p>You will notice that we do not have the Target CRM Guids for the Competitor and Opportunity here so we will use the Data Sync Lookup Function to provide this transformation.</p> <p>&nbsp;</p> <h3>Configure Data Source A</h3> <p>&nbsp;</p> <p>First connect Data Source A to the CSV File that contains the relationships.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_75d79187-bd11-4caf-8640-50be0f576c69"><img height="484" width="458" src="https://simegoimages.blob.core.windows.net/images/blog_2049965d-72ff-4ee4-8d33-0ff7d496a720" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> <p>&nbsp;</p> <p>Next we define a Lookup for Opportunity &ndash;&gt; CRM Opportunity.Name by adding a Lookup to the Opportunity Column linking to CRM Entity Opportunity and setting the Join Target column to the Name of the Opportunity.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_b2e5fb59-b0e0-4c23-a5b5-687a3fac433d"><img height="484" width="389" src="https://simegoimages.blob.core.windows.net/images/blog_dec11b1e-99ea-4b43-8fb3-fee5f2b40bb7" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> <p>&nbsp;</p> <p>Next we define a Lookup for Competitor&ndash;&gt; CRM Competitor.Name by adding a Lookup to the Competitor Column linking to CRM Entity Competitor and setting the Join Target column to the Name of the Competitor.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_4b9d1a06-f45d-4e84-a51b-444c6c89d394"><img height="484" width="390" src="https://simegoimages.blob.core.windows.net/images/blog_0ecb7abf-480a-4a8c-a5fa-8e338dc96e4c" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> <p>&nbsp;</p> <p>Now we can test the Lookups by creating a Schema Map of OpportunityId and CompetitorId from the Lookup Columns.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ebf1fe86-561e-4fc0-ac8e-e1f5d01e2872"><img height="484" width="569" src="https://simegoimages.blob.core.windows.net/images/blog_83fd9bfa-2374-41de-b511-395ba0b48e8b" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> <p>&nbsp;</p> <p>Then Check that the Data Looks good.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_29f871aa-a457-49d3-baf6-159a5e81384e"><img height="484" width="586" src="https://simegoimages.blob.core.windows.net/images/blog_95e092c1-2d5e-4e63-b1a0-4cab22cc0d82" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> <p>&nbsp;</p> <h3>Configure Data Source B</h3> <p>&nbsp;</p> <p>Connect Data Source B to the new &ldquo;Microsoft Dynamics CRM 2011 &ndash; N:N&rdquo; provider and choose the Relationship &ldquo;opportunitycompetitors_association&rdquo;.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ad477944-45c0-4f95-9efa-df3b5739c017"><img height="356" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_8ca5cd68-4ccc-4bf7-abb7-56cbbe5080e8" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> <p>&nbsp;</p> <p>Now configure your Schema Map so that you map the <strong>Lookup1_OpportunityId</strong> to <strong>OpportunityId</strong> and <strong>Lookup2_CompetitorId</strong> to <strong>CompetitorId</strong> and ensure that the Key Column is checked for both columns as were creating a Composite Key based on both values.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_6be8f9c5-f353-4349-93fb-aa1ec3fad4d8"><img height="128" width="649" src="https://simegoimages.blob.core.windows.net/images/blog_5a53519a-4bb2-4ede-ad81-fa31c3c589ce" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> <p>&nbsp;</p> <p>Now you can Compare and Sync to update CRM with the Data and when you check CRM you will see the Relationship is now configured.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_b2faac44-6c38-427d-b18b-23aafd2a1c2d"><img height="204" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_18332e04-c2da-4e72-a581-6040770519e3" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> Team Blog Fri, 31 Aug 2012 10:45:32 +0000 http://www.simego.com/Blog/2012/08/Manually-Setup-Ouvvi-Data-Sync-Team-Server info@simego.com (Sean Cleaver) Manually Setup Ouvvi : Data Sync Team Server <p>This guide shows how to setup the default Ouvvi Database manually from a Script which maybe your only option when installing the database onto a remote SQL Server.</p> <p>Ouvvi requires a Database on SQL2008, 2008R2 or 2012 and also runs with the free SQLEXPRESS editions. Your SQL Server must be configured to accept Standard SQL Accounts.</p> <p>&nbsp;</p> <p>When Ouvvi starts it first offers the automatic option and this will connect to your Database server using Integrated Security from the Logged on user of the Browser. This can fail against remote servers or if this user does not have the required permissions.</p> <p><em>Note: If you get an error trying to access the Ouvvi Web Page check that you have Windows Authentication Feature in IIS installed and activated on the Ouvvi Web Site.</em></p> <p>To configure manually, select the option to generate a SQL Script of the database.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_8b5c64f7-730b-48a5-a62d-39d91215fb4b"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_6dc47f78-2220-4553-a9e7-f116c4653d7b" width="550" height="484" /></a></p> <p>&nbsp;</p> <p>Now you need to enter some details for the script generation, specifically the SQL Database and SQL Username (SQL Stored Procedures will be granted Execute Permission with this Username) and the Ouvvi server that is running the Notification Service (Usually the current machine).</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f41fddd1-97c9-4223-a6a5-676a61383aed"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_4ab16817-5cfc-4946-8efb-756de8eeca00" width="551" height="484" /></a></p> <p>&nbsp;</p> <p>This will now create a SQL Script that you need to execute on your SQL Server to create the Database Structure and default Static Data.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_7e4e94bc-52da-4906-b4d7-889ba751e37f"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_bfafadba-1909-4bd2-baf4-23095a6239b4" width="551" height="484" /></a></p> <p>&nbsp;</p> <p>Now you need to create a configuration file so that Ouvvi can connect to your database.</p> <p>Create a new XML Text File called <strong>Ouvvi.config</strong> and place it in directory <strong>C:\ProgramData\Simego\Ouvvi</strong></p> <p>This File should contain the SQL Connection String to Ouvvi an example is shown below.</p> <pre> &lt;?xml version="1.0"?&gt;<br />&lt;configuration&gt;&nbsp; <br />&nbsp; &lt;connectionStrings&gt;&nbsp; &nbsp; <br />&nbsp; &nbsp; &lt;add name="ConsoleDB" connectionString="data source=.\SQLEXPRESS;initial catalog=Ouvvi;integrated security=false;persist security info=true;User ID=Ouvvi_User;Password=password;" /&gt;&nbsp; <br />&nbsp; &lt;/connectionStrings&gt;<br />&lt;/configuration&gt;<br /></pre> <p>You can then Click Home in your browser to take you back to Ouvvi home page.</p> <p>You may get a Schema Error where the database needs upgrading and by following the same procedure you can generate a Script to bring the Database up to date.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_024ff2e5-410f-4f9d-abbb-2da115c74b93"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_c5730db0-9e5e-4c51-9e99-3da2bca17840" width="551" height="484" /></a></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_e946996d-d7d3-4e5c-83ec-6086e858b7b5"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_54b5ddfc-e39a-4f4b-a28a-f31dcafe8429" width="551" height="484" /></a></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_c330b00f-4238-4a3f-a387-5e59aa83d156"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_eb6e2389-4b6a-481f-953c-94efdc470ad5" width="551" height="484" /></a></p> <p>&nbsp;</p> <p>Then once the Database is all configured and correct you can then access the Ouvvi Home Page. Which details the last few steps to start the Ouvvi Processing service and start creating projects.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_c9080447-9d2d-4ade-bfe7-08e151b7267a"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_39f18863-25b3-4627-acd0-b698344d24dc" width="551" height="484" /></a></p> Team Blog Wed, 15 Aug 2012 08:53:17 +0000 http://www.simego.com/Blog/2012/08/Workaround-API-Limitations-eg-CreatedBy-and-ModifiedBy info@simego.com (Sean Cleaver) Workaround API Limitations eg CreatedBy and ModifiedBy <p>This is a little trick you can use with Data Synchronisation Studio Project Automation Feature to workaround limitations with certain system API&rsquo;s such as Dynamics CRM.</p> <p>&nbsp;</p> <p>For example you can&rsquo;t directly update the CreatedOn, ModifiedOn fields in Microsoft CRM via the Web Service API. However if you ignore the warnings about modifying the Database directly you can implement a little function at the end of the Sync to update the Database directly.</p> <p>&nbsp;</p> <p>This example uses a slightly modified version of the <a href="https://github.com/robconery/massive/">Massive SQL Library by Rob Conery</a> to simplify the SQL required and make it really simple.</p> <p>&nbsp;</p> <p>Essentially at the end of the Synchronisation we run through the change set and update the fields in the Database Directly.</p> <p>&nbsp;</p> <p>In this example the CreatedOn and ModifiedOn fields are part of the Sync operation as Updating them doesn&rsquo;t error it simply ignores their values this way they are part of the Changed row data.</p> <p>&nbsp;</p> <p>The Data Source is simply a list of entityids, the CreatedOn Date and the ModifiedOn Date.</p> <p>&nbsp;</p> <p>Change the model details from Account and Accountid to the correct entityId.</p> <p>&nbsp;</p> <pre class="csharpcode"> <span class="kwrd">public</span> <span class="kwrd">override</span> <span class="kwrd">void</span> End(ProjectAutomationResult result) { <span class="kwrd">if</span> ( result.HasChanges &amp;&amp; result.Success ) { <span class="kwrd">string</span> connection = <span class="str">"Data Source=.;Initial Catalog=SimegoLtd_MSCRM;Integrated Security=True;"</span>; var model = <span class="kwrd">new</span> Simego.DataSync.Helpers.Massive.DynamicModel(connection, <span class="str">"Account"</span>, <span class="str">"AccountId", "System.Data.SqlClient"</span>); <span class="kwrd">foreach</span>(var update <span class="kwrd">in</span> result.CompareResult.Updated) { model.Update( <span class="kwrd">new</span> { CreatedOn = (DateTime)update.SourceRow[1].BeforeColumnValue, ModifiedOn = (DateTime)update.SourceRow[2].BeforeColumnValue }, (Guid)update.SourceRow[0].BeforeColumnValue); } } }</pre> <p>This trick could be used with many different systems to workaround limitations in the API. For example the Same trick could be used with SharePoint to update the Database directly this way too.</p> Team Blog Thu, 09 Aug 2012 11:43:28 +0000 http://www.simego.com/Blog/2012/08/Dynamics-CRM-Import-limits david@simego.com (David Evans) Dynamics CRM Import Limits <p>As &lsquo;Customer Relationship Solutions&rsquo; become more and more utilised in industry, IT departments are going to be migrating/importing more and more data. Dynamics CRM is an excellent CRM solution and as such will continue to be the focus of many import/export scenarios.</p> <p>The Dynamics CRM Import Wizard has several limits due to it &lsquo;uploading&rsquo; the source data and fetchsql results.</p> <ol> <li><a style="background-color: white;" href="http://julieyack.blogs.com/my_weblog/2012/06/importing-that-flat-data-will-be-easy-peasy-i-promise.html">Importing that flat data will be easy peasy, I promise!</a><span style="background-color: white;"> (this includes the reason and fix!)</span> </li> <li><a style="background-color: white;" href="http://community.dynamics.com/product/crm/crmnontechnical/b/crmlessons/archive/2011/06/16/crm-2011-data-import-wizard.aspx">CRM 2011 Data Import Wizard</a> </li> </ol> <p>I though I would just post a few screenshots to illustrate how easy it is to import and export data to and from Dynamics CRM 2011 (and 4.0) with Data Synchronisation Studio.</p> <p>What is more important is that we can change the source re-sync, ie make a mistake, fix and re-sync:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_189e55b5-70bb-40de-83be-3bf1e2cb327f"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" title="Screen Shot 2012-08-07 at 15.16.33" border="0" alt="Screen Shot 2012-08-07 at 15.16.33" src="https://simegoimages.blob.core.windows.net/images/blog_1ce5d3d9-8e8d-44d3-bf00-7062a11d01c3" width="630" height="377" /></a></p> <p>What may be more interesting is the speed:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_962e1cda-c3ae-4f80-8469-a8dd26fb4c76"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" title="Screen Shot 2012-08-07 at 15.31.19" border="0" alt="Screen Shot 2012-08-07 at 15.31.19" src="https://simegoimages.blob.core.windows.net/images/blog_e00c6c3b-2c47-49eb-b7ea-6c0313c6d604" width="630" height="380" /></a></p> <p>For a 35 Second video of developing the import from SQL to CRM:</p> <p><iframe height="500" src="http://www.youtube.com/embed/T-zNaYaCyk0" frameborder="0" width="700"></iframe></p> <div class="paddingTop">Watch as we sync from start to finish in 35 seconds! <div></div> </div> Team Blog Wed, 08 Aug 2012 12:02:35 +0000 http://www.simego.com/Blog/2012/08/CRM-Change-Trigger-Ouvvi-Data-Sync-Team-Server info@simego.com (Sean Cleaver) CRM Change Trigger-Ouvvi : Data Sync Team Server <p>We have created a CRM Entity change tracking trigger for Data Sync Team Server that operates in a similar way to the SharePoint List Change Trigger.</p> <p>Internally the trigger system calls each trigger at a 30 second interval to evaluate it&rsquo;s state, the CRM trigger will then call a Web Service in CRM to see when the last item was created or updated. This is a quick call that only returns a single data item the DateTime of the last change.&nbsp; If this has changed since the last time time a change was detected then any projects associated with this trigger will be started.</p> <p>Rather than synchronising on a timed schedule you can now achieve almost real-time updates (max 30 second delay) by implementing the Entity Change Trigger with your CRM environment.</p> <p>This feature is currently only supported with OnPremise AD Authenticated CRM 4.0/2011 installations.</p> <p>The feature requires no change to your CRM environment and can therefore be easily deployed.</p> <p>&nbsp;</p> <p>To configure the Trigger add a new Trigger of type &lsquo;Crm Entity Trigger&rsquo;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_864f1aec-4d58-4106-ab3e-25e1ac3070bc"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_89b51096-9061-40b8-b23f-908ad0bfecf0" width="644" height="446" /></a></p> <p>&nbsp;</p> <p>Then enter the URL, Organisation Name and Entity Name you want to monitor you may also need to enter some credentials if Ouvvi is not running under an account that has access to your CRM environment.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_2321bade-7b30-4deb-967d-ceedf2b8165f"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_e14733d0-83d9-414b-bcdc-d5651e4c4d3d" width="620" height="484" /></a></p> <p>&nbsp;</p> <p>The only other configuration item required is to ensure that your projects are configured for &rsquo;Single Instance&rsquo; so that multiple synchronisation operations do not occur in parallel.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_bc1b5e97-58a0-4d0e-85aa-b5fa16aace12"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_9f733e4a-568b-452a-8b58-fd6e83d718ca" width="644" height="190" /></a></p> <p>&nbsp;</p> <p>That&rsquo;s all it takes you can now simply associate this Trigger with any projects you want to run when a change is detected.</p> <p>&nbsp;</p> <p>This new feature is in release 2.0.226&nbsp;<a title="http://www.simego.com/Install/Ouvvi-Beta" href="http://www.simego.com/Install/Ouvvi-Beta">http://www.simego.com/Install/Ouvvi-Beta</a></p> Team Blog Tue, 07 Aug 2012 10:29:29 +0000 http://www.simego.com/Blog/2012/08/HTTP-Change-Trigger-Ouvvi-Data-Sync-Team-Server info@simego.com (Sean Cleaver) HTTP Change Trigger-Ouvvi : Data Sync Team Server <p>The latest Ouvvi Data Sync Team Server update includes a new Trigger for integration with custom solutions. This trigger calls an external HTTP service to evaluate whether the Trigger should be fired.</p> <p>The HTTP service should simply return the Date Time value of the last change in the format <strong>yyyy-MM-ddThh:mm:ss</strong>. Ouvvi will then use this value to determine whether the Trigger should fire. As this is a triggering system your service should return very quickly so that it does not tie up the internal Ouvvi trigger list.</p> <p>&nbsp;</p> <p>To configure this trigger simply add the &lsquo;Http Trigger&rsquo;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_2dcc525d-942b-4f83-bdf5-e10389344e6e"><img height="400" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_63cad9b3-9b50-4a57-a52a-10116b5b4052" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>&nbsp;</p> <p>Configure the URL and HTTP Method &lsquo;GET&rsquo; or &lsquo;POST&rsquo; if your service is authenticated you can enter Windows Credentials here as well.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_6842dcd0-ada8-4424-b361-3269364f397f"><img height="484" width="588" src="https://simegoimages.blob.core.windows.net/images/blog_2b7bb067-4a64-4cc2-a6f7-5b2853a66cd2" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>&nbsp;</p> <p>Now simply associate this trigger with any projects you want to start when the Trigger fires.</p> Team Blog Tue, 07 Aug 2012 10:07:04 +0000 http://www.simego.com/Blog/2012/07/Updating-Custom-Geckoboard-Widget info@simego.com (Sean Cleaver) Updating Custom Geckoboard Widget <p>This is a quick guide that demonstrates how to update a custom Widget on <a href="http://www.geckoboard.com/">Geckoboard</a> with the result of a Data Synchronisation using the Project Automation Feature.</p> <p>Note: Requires Data Synchronisation Studio Release 3.0.570 or greater.</p> <p>First you need an account at <a href="http://www.geckoboard.com/">Geckoboard</a> you can signup for a free trial to test it out.</p> <p>&nbsp;</p> <p>Once you have your first Dashboard add a Custom Widget and configure it for Push, here I selected the Text widget that were just going to push a Single Text value to.</p> <p>You need to make a note of the Push URL and get your API key form your Account that we need in Data Sync Later.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_567184b2-e8f9-4891-9ecf-8d91eba65636"><img height="493" width="659" src="https://simegoimages.blob.core.windows.net/images/blog_a2027862-974d-4b24-8419-0984157a04d7" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>&nbsp;</p> <p>Now that you have the widget on your dashboard we need to send it some data.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f987a518-1d21-431d-8bfd-d07e94c22330"><img height="305" width="322" src="https://simegoimages.blob.core.windows.net/images/blog_e4a19f37-b744-4fa3-b6d8-851459824f6f" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>&nbsp;</p> <p>This example uses a demo Data Sync project you can <a href="http://www.simego.com/downloads/GeckoUpdateSample.zip">download</a> but you can use any Data Source you like such as CRM, SharePoint, SQL etc.</p> <p>&nbsp;</p> <p>First we setup our Data Source and preview the Data from Data Source A.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_38cc5d9b-a630-4886-a3a2-b57071032d10"><img height="143" width="516" src="https://simegoimages.blob.core.windows.net/images/blog_f33fa878-c5f0-4c89-8eef-d8b3b7482c3f" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>This is a single item that were going to send to Geckoboard, next Export this result to an XML file from the Toolbar so we have a Datasource that we can connect Datasource B to and handle change events.</p> <p>&nbsp;</p> <p>Connect Datasource B to the .NET XML Dataset provider by browsing to the XML file we just created.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_bb1c74e6-3481-4705-b291-3f16c0d014cb"><img height="357" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_b85c8e34-2100-4a15-88fe-9b41f7400d59" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>&nbsp;</p> <p>You now have a simple Data Sync project set up that Updates the XML file when there are changes to the Source Data.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_1fbca46d-511e-4d2a-8cb4-efb541c6c402"><img height="334" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_68667bb8-2316-42f7-b874-3d10d22d0de0" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_55ea9215-6f5c-427a-b7e4-1f3ace1735e5"><img height="250" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_69411058-d695-4f2f-81ba-9502765fe7e9" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> <p>&nbsp;</p> <p>Now for the clever bit, switch to the Project Automation Tab and enable Project Automation so that we can send the results to Geckoboard.</p> <p>&nbsp;</p> <p>We need to write the code that generates the Json payload to send to Geckoboard this is quite simple with .NET Anonymous types.</p> <p>&nbsp;</p> <p>Within the End method of the Project Automation which occurs after the synchronisation to the XML file and therefore only updates Geckoboard when changes occur add the following code. <em>[Update widgetUrl and apiKey with your values].</em></p> <p>&nbsp;</p> <pre class="csharpcode"> <span class="kwrd">public</span> <span class="kwrd">override</span> <span class="kwrd">void</span> End(ProjectAutomationResult result) { <span class="kwrd">if</span> ( result.Success &amp;&amp; result.HasChanges ) { <span class="kwrd">string</span> widgetUrl = <span class="str">"https://push.geckoboard.com/v1/send/aaabbbccc"</span>; <span class="kwrd">string</span> apiKey = <span class="str">"aaabbbcccc"</span>; var data = <span class="kwrd">new</span> { api_key = apiKey, data = <span class="kwrd">new</span> { item = <span class="kwrd">new</span> [] { <span class="kwrd">new</span> { text = result.CompareResult.Updated[0].SourceRow[1].BeforeColumnValue, <span class="kwrd">value</span> = result.CompareResult.Updated[0].SourceRow[0].BeforeColumnValue } } } }; var client = <span class="kwrd">new</span> System.Net.WebClient(); client.UploadString(widgetUrl, Json.Encode(data)); } }</pre> <p>&nbsp;</p> <p>The CompareResult object contains the results of the Comparison and in this case were only interested in Updates. the Updated object is an Array of Updated Rows and the SourceRow object links back to the Datasource A row. The index of the Columns match the Schema Map layout.</p> <p>&nbsp;</p> <p>And then after the Sync you will see your Geckoboard updated.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f2dae647-a47e-4e7d-b662-e6a6a06d0696"><img height="271" width="278" src="https://simegoimages.blob.core.windows.net/images/blog_70138729-04a4-481d-b2ae-5790c4c18fe2" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> Team Blog Fri, 27 Jul 2012 10:57:25 +0000 http://www.simego.com/Blog/2012/07/Import-File-Share-or-Folder-into-SharePoint-2010-Document-Library info@simego.com (Sean Cleaver) Import File Share or Folder into SharePoint 2010 Document Library <p>To setup a File/Folder/Share Synchronisation to a SharePoint 2010 document library is quick and easy with Data Synchronisation Studio.</p> <p>&nbsp;</p> <p>Connect Data source A to your Source Folder and ensure you set the property &ldquo;WebFriendlyPaths&rdquo; to <strong>True</strong>.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_4da1021c-8936-4bbb-9efb-00577b30699c"><img height="356" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_1597101d-adb5-43cc-b7a3-42a442c4a496" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>&nbsp;</p> <p>Connect your Target Data source B to SharePoint using the SharePoint <strong>Client API</strong> provider this is for SharePoint 2010 only.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_8389d324-d9f0-45b3-bb90-945fdcfd4536"><img height="356" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_183f08f4-3e36-45d2-860c-c04db72a4f52" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>&nbsp;</p> <p>Map your Source to your Target like this</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_269e25c3-355a-42bb-b882-62497ff4e59d"><img height="156" width="714" src="https://simegoimages.blob.core.windows.net/images/blog_129c2a33-3934-4c53-896a-627efeaa31c6" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>&nbsp;</p> <p>And that&rsquo;s it Compare A-&gt;B and then Synchronise to copy the files from your Folder/Share into your SharePoint Document Library. Even the Created and Modified fields will match the File timestamps.</p> <p>&nbsp;</p> <p>If you want to update Metadata columns at the same time you can by either using Dynamic columns to return values for the Metadata Columns or use a Lookup function against another Data Source to provide the values for your metadata.</p> Team Blog Wed, 25 Jul 2012 14:15:01 +0000 http://www.simego.com/Blog/2012/07/Transform-Any-Data-with-Data-Sync info@simego.com (Sean Cleaver) Transform Any Data with Data Sync <p>We have a database with order information where we just have 2 columns <strong>ID</strong> and <strong>Data</strong> where the Data column holds a Blob of strings like this below. I wanted to extract these strings into separate columns so that they could be stored in the database.</p> <p>&nbsp;</p> <p><span style="font-family: 'Courier New'; font-size: small;" face="Courier New" size="2">FIRST_NAME:Sean <br />LAST_NAME:Cleaver <br />COMPANY:Simego Ltd <br />EMAIL:sean@simego.com <br />PHONE: <br />FAX: <br />COUNTRY:United Kingdom <br />STATE:East Susses <br />CITY:St Leonards-on-Sea <br />ZIP:TN38 9NP <br />ADDRESS:34 Stirling Road <br />LICENSE_CODE:5E366AAFBB <br />EXPIRATION_DATE:2012-06-23 <br />DATE_UPDATED: <br />CHANGED_BY:VENDOR <br />LICENSE_TYPE:REGULAR <br />DISABLED:0 <br />RECURRING:0a <br />LICENSE_PRODUCT:4559088 <br />START_DATE:2012-05-23 <br />LICENSE_LIFETIME:0 <br />PARTNER_CODE: <br />PSKU: <br />ACTIVATION_CODE:DCDA2M-MEX6DR-Q <br />EXPIRED:0 <br />HASH:7b9dfb715ae1a703e12603aa</span></p> <p>&nbsp;</p> <p>With Data Sync it took about 5 minutes to write a few Dynamic Columns to transform this into the right shape to import virtually anywhere. (CRM, Sharepoint, SQL etc)</p> <p>&nbsp;</p> <p>For each row we basically read each line creating a Dictionary of the Values and then update the Dynamic Properties with the values from the Dictionary. The main task here was Cut &amp; Paste <img src="https://simegoimages.blob.core.windows.net/images/blog_e68e05b8-e0fa-483b-b3be-4a4972257bbe" alt="Smile" class="wlEmoticon wlEmoticon-smile" style="border-style: none;" /></p> <pre class="csharpcode"> <span class="kwrd">public</span> <span class="kwrd">string</span> FIRST_NAME { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> LAST_NAME { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> COMPANY { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> EMAIL { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> PHONE { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> FAX { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> COUNTRY { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> STATE { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> CITY { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> ZIP { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> ADDRESS { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> LICENSE_CODE { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> EXPIRATION_DATE { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> DATE_UPDATED { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> CHANGED_BY { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> LICENSE_TYPE { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> DISABLED { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> RECURRING { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> LICENSE_PRODUCT { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> START_DATE { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> LICENSE_LIFETIME { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> PARTNER_CODE { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> PSKU { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> ACTIVATION_CODE { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> EXPIRED { get; set; } <span class="kwrd">public</span> <span class="kwrd">string</span> HASH { get; set; } <span class="kwrd">public</span> <span class="kwrd">override</span> <span class="kwrd">bool</span> BeginRow() { Dictionary&lt;<span class="kwrd">string</span>, <span class="kwrd">string</span>&gt; items = <span class="kwrd">new</span> Dictionary&lt;<span class="kwrd">string</span>, <span class="kwrd">string</span>&gt;(); <span class="kwrd">using</span>(var sr = <span class="kwrd">new</span> StringReader(Data)) { <span class="kwrd">string</span> line = <span class="kwrd">string</span>.Empty; <span class="kwrd">while</span>((line = sr.ReadLine()) != <span class="kwrd">null</span>) { <span class="kwrd">string</span> [] split = line.Split(<span class="str">':'</span>); <span class="kwrd">if</span> ( split.Length == 2 ) { items.Add(split[0], split[1]); } } } FIRST_NAME = items[<span class="str">"FIRST_NAME"</span>]; LAST_NAME = items[<span class="str">"LAST_NAME"</span>]; COMPANY = items[<span class="str">"COMPANY"</span>]; EMAIL = items[<span class="str">"EMAIL"</span>]; PHONE = items[<span class="str">"PHONE"</span>]; FAX = items[<span class="str">"FAX"</span>]; COUNTRY = items[<span class="str">"COUNTRY"</span>]; STATE = items[<span class="str">"STATE"</span>]; CITY = items[<span class="str">"CITY"</span>]; ZIP = items[<span class="str">"ZIP"</span>]; ADDRESS = items[<span class="str">"ADDRESS"</span>]; LICENSE_CODE = items[<span class="str">"LICENSE_CODE"</span>]; EXPIRATION_DATE = items[<span class="str">"EXPIRATION_DATE"</span>]; DATE_UPDATED = items[<span class="str">"DATE_UPDATED"</span>]; CHANGED_BY = items[<span class="str">"CHANGED_BY"</span>]; LICENSE_TYPE = items[<span class="str">"LICENSE_TYPE"</span>]; DISABLED = items[<span class="str">"DISABLED"</span>]; RECURRING = items[<span class="str">"RECURRING"</span>]; LICENSE_PRODUCT = items[<span class="str">"LICENSE_PRODUCT"</span>]; START_DATE = items[<span class="str">"START_DATE"</span>]; LICENSE_LIFETIME = items[<span class="str">"LICENSE_LIFETIME"</span>]; PARTNER_CODE = items[<span class="str">"PARTNER_CODE"</span>]; PSKU = items[<span class="str">"PSKU"</span>]; ACTIVATION_CODE = items[<span class="str">"ACTIVATION_CODE"</span>]; EXPIRED = items[<span class="str">"EXPIRED"</span>]; HASH = items[<span class="str">"HASH"</span>]; <span class="kwrd">return</span> <span class="kwrd">true</span>; <span class="rem">// return false to remove row from results.</span> }</pre> <p>&nbsp;</p> <p>And the results in Data Sync Data Preview</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_409d68fd-2e99-4b2e-8bb7-11ae6aa65348"><img height="279" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_38a0950e-433e-4ab5-966d-695c96662d16" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> Team Blog Tue, 17 Jul 2012 17:55:49 +0000 http://www.simego.com/Blog/2012/07/Duplicate-Key-Values-Behaviour-Change info@simego.com (Sean Cleaver) Duplicate Key Values Behaviour Change <p>Data Sync Release: 3.0.568</p> <p>We have an update to the way Data Sync handles Duplicate Key values, previously during the load when Data Sync hit a duplicate key value it caused an internal exception and we displayed a list of keys showing you the exception. You would then need to either adjust the Key column in the schema map or fix the data to ensure all rows were unique.</p> <p>&#160;</p> <p>&#160;<a href="https://simegoimages.blob.core.windows.net/images/blog_7279c1fb-15f5-4399-83a3-d4ac0013516d"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_f789327a-eec1-4fea-8115-ea63fecbec5c" width="581" height="574" /></a></p> <p>&#160;</p> <p>This would keep happening for every duplicate key in your data source and it could be quite time consuming correcting the data to ensure that the key values are unique.</p> <p>&#160;</p> <p>We have now changed this behaviour so that duplicate keys no longer cause the application to stop but rather the duplicate key rows are excluded from the comparison data set and flagged as a duplicate.</p> <p>When you preview the Data from the Schema Map PreviewA/B functions you will see a error icon next to the duplicate rows and a message at the bottom indicating how many duplicates there are in the result set. This provides a quick visual indication to whether your Key selection is correct.</p> <p>&#160;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_0ad344b7-13d5-47a0-b70c-2ee3ae6cd017"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_4db37761-79b1-4281-a730-4071ae6d00da" width="644" height="423" /></a></p> <p>&#160;</p> <p>likewise the comparison results will show that some rows were excluded but it will not prevent the comparison from running.</p> <p>&#160;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_915c743b-89d2-49f4-9ff9-b956bbb7a981"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_f5972554-d575-440a-94e3-94534a632735" width="615" height="590" /></a></p> <p>&#160;</p> <p>It is possible to force an error if there are duplicates via the Project Automation feature. You can simply test for Duplicates before the Compare and throw an exception to stop the process like this below.</p> <p>&#160;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_642a4007-c615-441a-bc57-baab8cb9e995"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_bfc28a81-8398-4e83-9ca2-97373a00a730" width="644" height="401" /></a></p> <p>&#160;</p> <p>When you run the compare with this check Data Sync will simply show the error or log the error when running from the Run Tool.</p> <p>&#160;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_7f9c1ee4-bde8-47f9-892c-6153ebdaceb7"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: block; float: none; margin-left: auto; border-top: 0px; margin-right: auto; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_d4892fcb-7a8a-422b-99d8-9f4219a828db" width="507" height="359" /></a></p> Team Blog Mon, 16 Jul 2012 12:47:40 +0000 http://www.simego.com/Blog/2012/07/Partner-Program info@simego.com (Sean Cleaver) Partner Program <p>Are you an Application Integrator deploying solutions with Microsoft Dynamics or SharePoint?<span style="background-color: white;">&nbsp;</span></p> <p>We're very pleased to announce the new Simego&rsquo;s Partner Programme. This allows qualified partners to earn money by packaging their integration consultancy with Simego&rsquo;s products?</p> <p>To learn more and find out if you are already qualified <a title="Ask a question" href="/Question/">contact us</a> today.</p> Team Blog Fri, 13 Jul 2012 13:14:33 +0000 http://www.simego.com/Blog/2012/07/Data-Synchronisation-Studio-Fails-to-Start info@simego.com (Sean Cleaver) Data Synchronisation Studio-Fails to Start <p>If your copy of Data Synchronisation Studio fails to open the main window when you run the program it could be that the window location file is corrupt.</p> <p>The easiest thing to do is to remove your local settings for Data Sync and let it start from a clean start.</p> <p>Delete the following directory replacing &lsquo;Sean&rsquo; for your profile name.</p> <p>C:\Users\Sean\AppData\Local\Simego\DataSyncStudio30</p> <p>Now when Data Sync starts it will start with default settings.</p> <p><strong>Note:</strong> This directory maybe hidden on your system so you might need to show hidden files.</p> Team Blog Tue, 10 Jul 2012 14:27:37 +0000 http://www.simego.com/Blog/2012/06/Yammer-Integration-with-Sharepoint david@simego.com (David Evans) Yammer Integration with Sharepoint <p>Yammer stands to make an even bigger splash in the Enterprise sector with Microsoft leading it. At Simego we have always developed solutions that allowed IT staff to integrate their systems with ease and Yammer will be no exception.</p> <p>All scenarios DO NOT NEED Yammer Admin permissions</p> <h3>Scenario: Send an update Notification to Yammer when a Status Changes in a Sharepoint list</h3> <p>Many IT departments need to notify their users when there are issues with email, file systems, Line-of-Business apps etc. This scenario shows how to connect to Yammer, connect to a Sharepoint &lsquo;Status Board&rsquo; and respond to changes in the list by sending a notification to Yammer.</p> <p>Prerequisites:</p> <ol> <li>A Yammer account for your service to talk through : <a href="mailto:systems@yourcomany.com">systems@yourcomany.com</a> </li> <li>Data Sync Studio <a href="http://www.simego.com/Install/Data-Synchronisation-Studio">Download a 7 day full Version trial</a></li> <li>Sharepoint 2007 &amp; 2010 (Online or On-Premise) </li> <li>MS SQL Express (or any to compare the list for changes). </li> </ol> <h5>Step 1: Create a new List in Sharepoint similar to the one below</h5> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f7d31319-699d-4fb2-8e08-31df10e771be"><img height="471" width="630" src="https://simegoimages.blob.core.windows.net/images/blog_a4c3fef5-274a-4720-8236-6680056b2d1b" alt="Sharepoint IT Systems Status List" border="0" title="Sharepoint IT Systems Status List" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>This is essentially a &lsquo;Status Board&rsquo; for the IT Department. Each &lsquo;Service&rsquo; can have the following states:</p> <ol> <li>Operating with no issues</li> <li>The service is back up for most, but some users may be experiencing issues </li> <li>We are experiencing a serious problem with this service </li> <li>A small percentage of users are experiencing issues</li> </ol> <p>We want these Status updates to appear in Yammer</p> <h5>Step 2: Open Data Sync and connect to the list created in Step 1</h5> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_b36c6c42-79c3-4b3f-8126-29cccbfcf617"><img height="492" width="630" src="https://simegoimages.blob.core.windows.net/images/blog_be890524-7ed5-493a-a626-d42ae0760edc" alt="Step 1 Connect to Sharepoint Status Baord" border="0" title="Step 1 Connect to Sharepoint Status Baord" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_33a4df5d-aa94-47aa-b757-5e788f84dab7"><img height="492" width="630" src="https://simegoimages.blob.core.windows.net/images/blog_f5dffdfc-8717-4c7c-8fc9-ea870b3cfc3b" alt="Step 1 Select the Status List" border="0" title="Step 1 Select the Status List" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <h5>Step 3: Sync the SharePoint list with a SQL List (Data Sync can create the SQL Table for you)</h5> <p>Data Sync has a feature that left you create a new SQL Table based on the contents of your source. We are going to use that as a target. Create a SQL Table using the wizard from the Tools Menu.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_3eff256d-6edb-4874-902f-7d26a12766f6"><img height="492" width="630" src="https://simegoimages.blob.core.windows.net/images/blog_1c6ddd70-45bf-4124-98e6-6a36da089375" alt="Step 2 Create the SQL Sync Table" border="0" title="Step 2 Create the SQL Sync Table" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>You can now compare the Sharepoint List to the SQL Table and Synchronise by pressing the :</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_6c0eac6a-b127-46fc-aaaa-a88cbbeb6d0b"><img height="27" width="110" src="https://simegoimages.blob.core.windows.net/images/blog_7c361756-319a-4b58-968c-25b49ae4d0c1" alt="Compare A to B" border="0" title="Compare A to B" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>and then</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_49a89d17-c325-42d6-9d86-f9fb7bdf622d"><img height="25" width="96" src="https://simegoimages.blob.core.windows.net/images/blog_5e00ff8a-0f96-4b85-ab2a-e73774e93134" alt="Synchronse A to B" border="0" title="Synchronse A to B" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>&hellip;this will synchronise all the items in the Sharepoint List into the SQL Table.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_5d2f9fae-ad01-42ff-9190-b046ab5d8398"><img height="494" width="630" src="https://simegoimages.blob.core.windows.net/images/blog_f667bc15-8a29-4c2d-9389-91723503129d" alt="Step 2 Sync the Status into the SQL" border="0" title="Step 2 Sync the Status into the SQL" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>So, if you change an item in the Sharepoint list and then Compare and Synchronise a single update will be make to the SQL Table.</p> <h3><em>It would be so useful to send that change notification to Yammer!!</em></h3> <p>Luckily for us Data Sync is unbelievable flexible, and uses the .net Framework with a built in development environment to hook into synchronisation events. (Sounds hard but it is automagic).</p> <h5>Step 4: Enable &lsquo;Project Automation&rsquo;</h5> <p>On the &lsquo;Project Automation&rsquo; Tab click on the &lsquo;Enable Project Automation&rsquo; link. <a href="https://simegoimages.blob.core.windows.net/images/blog_1c3d6971-0533-4b76-9468-9330c65749e3"><img height="560" width="630" src="https://simegoimages.blob.core.windows.net/images/blog_4b30ba6c-573d-42ff-92b1-46879ed3ccd5" alt="Project Automation" border="0" title="Project Automation" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>This is the bit that makes or breaks any web based solution like Yammer&hellip;.Integration.</p> <h5>Step 5: Set up an Application in Yammer.</h5> <p>Waltz on over to Microsoft Yammer&rsquo;s application creation page:</p> <p><a href="http://www.yammer.com/client_applications/new" title="http://www.yammer.com/client_applications/new">http://www.yammer.com/client_applications/new</a></p> <p>and complete the following details obviously replacing your fake name for Simego&rsquo;s!</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_715cfafd-f00c-4df8-9fd7-b83a16d9ed54"><img height="576" width="571" src="https://simegoimages.blob.core.windows.net/images/blog_8116dfd9-5482-4fe2-8f3f-04c70f94204b" alt="Register a new Yammer Application" border="0" title="Register a new Yammer Application" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>and you should get a set of OAuth Keys and Secrets:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d87c59d5-a785-4e53-889c-0f30534543d0"><img height="551" width="577" src="https://simegoimages.blob.core.windows.net/images/blog_e22a0b27-6168-4da0-a314-eb229901377e" alt="Results of Yammer Application Registration" border="0" title="Results of Yammer Application Registration" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>Data Sync has a built in solution for authenticating with Yammer through their standard protocols but it is never going to be plain sailing!</p> <h5>Step 6: Do a Song and Dance to get OAuth working</h5> <h5>If Yammer approve &lsquo;Data Sync&rsquo; as a Global Application you will not need to do such a Song and Dance to get all these keys (Hint! Hint!)</h5> <p>We are going to use Data Sync&rsquo;s development environment to do a neat trick for us by doing the OAuth dance and handing out the secure tickets so we can post to Yammer!</p> <p>Back to Data Sync and to the Project Automation editor window. We are now going to use the Consumer Key and Consumer Secret to get a Token Key and Token Secret using DataSync&rsquo;s YammerHelper:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_604c724f-7210-4408-9ff1-30ca76d9aac9"><img height="506" width="570" src="https://simegoimages.blob.core.windows.net/images/blog_9a6b1193-d01a-40e1-9383-79be9df27cc9" alt="Project Automation Yammer OAuth Dance" border="0" title="Project Automation Yammer OAuth Dance" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>private Simego.DataSync.Providers.OAuth.YammerHelper yammer; <br />&nbsp;&nbsp;&nbsp; public override void Start() <br />&nbsp;&nbsp;&nbsp; { <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; //Configure the helper <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; yammer = new Simego.DataSync.Providers.OAuth.YammerHelper( <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; "OHdHeOvWcpODC69Q58ecg", //Consumer Key <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; "GbXNAYk8YwTTNDal4ZHwo6Pviph0nzJ3hPkhA4v5w"//Consumer Secret <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ); <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; //Get the yammer helper to negotiate with Yammer for our OAuth Keys <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Trace.WriteLine(yammer.Authorise()); <br />&nbsp;&nbsp;&nbsp; }</p> <p>Click the &lsquo;Compare A-B&rsquo; to start the OAuth negotiation.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_e9a14be1-bc4f-4ca1-8046-34896402e3e1"><img height="27" width="110" src="https://simegoimages.blob.core.windows.net/images/blog_f7d53d23-07b0-4aa4-b08f-b840f91a0747" alt="Compare A to B" border="0" title="Compare A to B" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>Your web browser will launch and as you to authorise the application <em>(Pro-Tip : If you are not signed in to Yammer then you have to sign-in on this screen)</em></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d74659d1-62cc-4606-b814-4cfc34b2fe9f"><img height="320" width="621" src="https://simegoimages.blob.core.windows.net/images/blog_0d611cf5-46bc-4256-bfc0-2d901a406190" alt="Yammer OAuth Authorise Window" border="0" title="Yammer OAuth Authorise Window" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>Click the &lsquo;Authorize&rsquo; button to progress to the next stage. You should be presented with a code number :</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_c5482991-de45-4244-a7ca-a8b3c3519102"><img height="281" width="626" src="https://simegoimages.blob.core.windows.net/images/blog_2e2451d5-6c10-4015-b338-4055319c0e9c" alt="Yammer OAuth Application Acceptance Key Window" border="0" title="Yammer OAuth Application Acceptance Key Window" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>Data Sync should have shown you a little dialog to enter this code into:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_1c966e17-3cfb-48ab-bb31-de0905e9ddb8"><img height="191" width="368" src="https://simegoimages.blob.core.windows.net/images/blog_2ba4c887-f9d5-42c0-a888-a90f222c3813" alt="Data Sync OAuth Code Application Entry" border="0" title="Data Sync OAuth Code Application Entry" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>The magic happens in the background and back on the Data Sync Project Automation page in the bottom of the window Data Sync will be displaying the TokenKey and TokenSecret:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_ef34ebdc-baa1-4e4b-ab5c-c04f88162fd8"><img height="536" width="630" src="https://simegoimages.blob.core.windows.net/images/blog_3d79abc4-c59b-4251-ab0d-b2210c7fa863" alt="Yammer Keys with OAuth Song and Dance Complete" border="0" title="Yammer Keys with OAuth Song and Dance Complete" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>Next we just need to copy those keys next to the Consumer Keys and Secrets and remove the yammer.Authorise() because we have completed the Dance!</p> <p>We can even send a Test Message using the yammer.SendMessage() method!</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_668b3683-3764-4900-8243-89510b4fc2ea"><img height="538" width="627" src="https://simegoimages.blob.core.windows.net/images/blog_5af5ace6-a0e1-4483-93c9-8069b65b26e5" alt="Yammer - Data Sync Connected!" border="0" title="Yammer - Data Sync Connected!" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <p>In Yammer we will see the message:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d7c89372-9acb-401e-a24c-304980ec4588"><img height="87" width="630" src="https://simegoimages.blob.core.windows.net/images/blog_5fb803a6-f3f2-4e38-9414-cf163f9a6812" alt="Yammer update of Data Sync Message" border="0" title="Yammer update of Data Sync Message" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" /></a></p> <h5>Step 7: Publish Changes when the Status Changes</h5> <p>All we need to do is add a little code to the &lsquo;End&rsquo; method:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_08e7319c-d837-4863-828f-b2c9ffe447b6"><img height="541" width="630" src="https://simegoimages.blob.core.windows.net/images/blog_49eaba74-f405-44d6-94e6-7f51cb9f881c" alt="Yammer Notification from Change" border="0" title="Yammer Notification from Change" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> <p>public override void End(ProjectAutomationResult result) <br />&nbsp;&nbsp;&nbsp; { <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if ( result.Success &amp;&amp; result.HasChanges ) { <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; foreach(var item in result.CompareResult.Updated) { <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; yammer.SendMessage(string.Format("{0}, {1}", <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; item.SourceRow[3].BeforeColumnValue, <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; item.SourceRow[2].BeforeColumnValue));&nbsp;&nbsp;&nbsp; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; } <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; } <br />&nbsp;&nbsp;&nbsp; }</p> <p><em>Note: The SourceRow columns are the columns in the Data Sync Sharepoint results referenced by number starting with 0.</em></p> <p>And we are Done!!</p> <h5>Step 8: Test</h5> <p>To test just update the status of one of your Systems in the Sharepoint List and the changes with appear in Yammer:</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f44c8315-2c61-4863-9a8f-ecffc1d696d6"><img height="93" width="630" src="https://simegoimages.blob.core.windows.net/images/blog_8f946629-7439-4bc6-ba9f-aeb5484510f7" alt="Yammer Serice Update from Data Sync" border="0" title="Yammer Serice Update from Data Sync" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> Team Blog Tue, 26 Jun 2012 09:09:11 +0000 http://www.simego.com/Blog/2012/06/Data-Sync-Project-as-a-Data-Source info@simego.com (Sean Cleaver) Data Sync Project as a Data Source <p>Imagine you have a reasonably complex Data Source configured that uses Dynamic Columns and possibly Lookups to define your Source Data Transformations. Currently if you wanted to use this again in another Sync job then you would have to copy it all again which isn’t ideal.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_1d7752b7-9b19-4177-9ecc-c69e35a0febb"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_f1d10ad4-c8f2-4116-9de8-3d5c3b07d4c4" width="535" height="484" /></a></p> <p>But now …with the new Data Sync Project File provider you can reference an external Data Sync Project as a Data Source.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d85d8fc1-8a99-4eaa-9b44-b9f1f7b2b942"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_c35e371e-8f0a-447f-9b5a-9aa64c7b4896" width="644" height="356" /></a></p> <p>So this Source appears just like a normal Data Source where you can then create new Dynamic Columns or Lookups against the already Dynamic Source.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_13c831a1-7444-4a76-8124-c5204d5bc0ed"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_b747f6ce-d4f6-46fc-9824-befd1be8c3dd" width="644" height="457" /></a></p> <p>You can see from this that you could easily define a set of Data Sources in Data Sync Project Files and then simply attach them to new projects.</p> <p>This new feature is in Data Sync Release 3.0.560</p> Team Blog Wed, 20 Jun 2012 14:34:37 +0000 http://www.simego.com/Blog/2012/06/Importing-Currency-Rates-from-Avangate info@simego.com (Sean Cleaver) Importing Currency Rates from Avangate <p>If you use Avangate as your payment processing partner you may need to download their current currency rates so that you can present the same rate conversion to your customers on&#160; your site.</p> <p>Avangate implement a simple XML based API where the following URL returns an XML Document containing the rates.</p> <p><a title="https://secure.avangate.com/content/exchange-xml.php?CURRENCY=USD" href="https://secure.avangate.com/content/exchange-xml.php?CURRENCY=USD">https://secure.avangate.com/content/exchange-xml.php?CURRENCY=USD</a></p> <p>With Data Synchronsiation Studio you can implement a Synchronisation Import job to import this in about 2 minutes.</p> <p>Simply connect with the XML Provider and Specify the URL and Item XML XPath of <strong>//currency</strong></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d421f1bd-ff3a-4fca-8152-82561627e5a1"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_0e0e76ac-d22e-4cad-a0e6-033a7c05e9a7" width="644" height="356" /></a></p> <p>Add 2 Columns <strong>csymbol</strong> and <strong>crate</strong></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f421d86d-731d-4182-a57b-30a87684d85d"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_66880480-ac34-4aa1-a80e-2e6d479e1b85" width="511" height="376" /></a></p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_1cfa52cb-2936-455a-88bc-de307029a982"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_5caf9c7a-ad17-47b1-9a58-f167a8dc669b" width="511" height="376" /></a></p> <p>And that is it preview the Data and setup the Synchronisation job to your Database.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_6ba53590-636e-477a-949e-71becb474621"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_d7aa5679-3123-450b-b1fd-290a9fbbcf8b" width="350" height="888" /></a></p> Team Blog Wed, 13 Jun 2012 14:16:01 +0000 http://www.simego.com/Blog/2012/06/Using-Automation-to-Send-Email info@simego.com (Sean Cleaver) Using Automation to Send Email <p>With the new Automation feature of Data Synchronisation Studio it is now incredibly easy to write custom actions in the pipeline. For example suppose you wanted an email sent whenever a synchronisation has run and you want to know the changed counts.</p> <p>You simply implement the End method of the Automation Class and send an SMTP Email message like this.</p> <p>&nbsp;</p> <pre class="csharpcode"> <span class="kwrd">public</span> <span class="kwrd">override</span> <span class="kwrd">void</span> End(ProjectAutomationResult result) { <span class="kwrd">if</span> ( result.Success &amp;&amp; result.HasChanges ) { var smtpClient = <span class="kwrd">new</span> System.Net.Mail.SmtpClient(<span class="str">"smtp.gmail.com"</span>, 587); smtpClient.EnableSsl = <span class="kwrd">true</span>; smtpClient.Credentials = <span class="kwrd">new</span> System.Net.NetworkCredential(<span class="str">"username"</span>, <span class="str">"password"</span>); <span class="rem">//Message to Send</span> <span class="kwrd">string</span> message = <span class="kwrd">string</span>.Format(<span class="str">"New {0}, Updated {1}, Deleted {2}"</span>, result.CompareResult.New.Count, result.CompareResult.Updated.Count, result.CompareResult.Deleted.Count); <span class="rem">//Send Message</span> smtpClient.Send(<span class="str">"from@simego.com"</span>, <span class="str">"to@simego.com"</span>, <span class="str">"Synchronisation Result"</span>, message); } }</pre> Team Blog Wed, 13 Jun 2012 11:45:11 +0000 http://www.simego.com/Blog/2012/06/Data-Synchronisation-Studio---New-Features info@simego.com (Sean Cleaver) Data Synchronisation Studio : New Features <p>With the release of Data Sync 3.0.550 we have a few new features to tell you about.</p> <h3>Integrated Screen Sharing</h3> <p>Form the Help Menu you can now initiate a Screen Sharing session with us to help you with issues that you may have. Please contact support first if you have a problem and we will see how we can help.</p> <h3>Project Automation</h3> <p>With Data Sync you can now hook the internal process to handle certain events as they happen. This allows you to call another Data Sync project of example or report the actual individual data changes.</p> <pre class="csharpcode"> <span class="kwrd">public</span> <span class="kwrd">override</span> <span class="kwrd">void</span> Start() { } <span class="kwrd">public</span> <span class="kwrd">override</span> <span class="kwrd">void</span> BeforeCompare(ProjectAutomationLoadResult loadResult) { } <span class="kwrd">public</span> <span class="kwrd">override</span> <span class="kwrd">void</span> BeforeSynchronisation(ProjectAutomationCompareResult compareResult) { } <span class="kwrd">public</span> <span class="kwrd">override</span> <span class="kwrd">void</span> End(ProjectAutomationResult result) { } <span class="kwrd">public</span> <span class="kwrd">override</span> <span class="kwrd">void</span> Error(ProjectAutomationException e) { }</pre> <p>&nbsp;</p> <h3>Custom Assemblies</h3> <p>If you want to use external assemblies that are not part of Data Sync with Dynamic Columns you can now tell Data Sync to load these Assemblies from the Project Properties menu.</p> <h3>Start a Synchronisation from Code</h3> <p>You can now easily start a Synchronisation Operation directly from your code like this.</p> <pre class="csharpcode">var op = <span class="kwrd">new</span> SynchronisationOperation(); op.Execute(<span class="str">@"C:\Temp\CSV.dsprj"</span>);</pre> <p>If your sync is 2-way then you need a little more</p> <pre class="csharpcode">var op = <span class="kwrd">new</span> SynchronisationOperation(); op.Key = <span class="str">"AAA"</span>; op.SolutionName = <span class="str">"CSV"</span>; op.SolutionPath = <span class="str">@"C:\Temp"</span>); op.Execute(<span class="str">@"C:\Temp\CSV.dsprj"</span>);</pre> <p>Then if you want to control logging you can provide your own implementation for IDataSourceLog</p> <h3>Load a Data Source from an existing Project</h3> <p>In dynamic columns you may need to load data from a remote source to build some kind of Lookup or other Transformation. Rather than accessing the underlying objects you can now easily load the data source from a pre-configured Data Sync Project file.</p> <pre class="csharpcode">var store = DataTableStore.LoadFrom(<span class="str">@"C:\Temp\CSV.dsprj"</span>); <span class="kwrd">foreach</span>(var row <span class="kwrd">in</span> store.Rows) { var id = (<span class="kwrd">int</span>)row[<span class="str">"ID"</span>]; }</pre> <p>You can also create a Dictionary from this data source easily like this</p> <pre class="csharpcode">var dict = store.ToDictionary&lt;<span class="kwrd">int</span>&gt;(<span class="str">"ID"</span>);</pre> <h3>Ouvvi</h3> <p>Ouvvi release 2.0.220 has also been updated to support these new Data Sync features.</p> Team Blog Fri, 08 Jun 2012 11:34:31 +0000 http://www.simego.com/Blog/2012/05/Updating-Active-Directory-Manager-Attribute info@simego.com (Sean Cleaver) Updating Active Directory Manager Attribute <p>With Data Synchronisation Studio you can now update the Manager attribute with the AD Provider if you modify the Provider configuration slightly.</p> <p>&nbsp;</p> <p>By default we parse out the Manager field so that it&rsquo;s more suitable for input to SharePoint or something. However you can now switch off this parsing to get to the raw AD attribute value.</p> <p>&nbsp;</p> <p>To update the manager you need to supply the full Distinguished name value of the AD Object you want to set which would be something like this below.</p> <p>&nbsp;</p> <p>CN=Sean Cleaver,CN=Users,DC=xxx,DC=xxx,DC=com</p> <p>&nbsp;</p> <p>Disable the <strong>TryParseValue</strong> and <strong>ReadOnly</strong> properties of the <strong>Manager</strong> field in the Property Collection on the AD Provider.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_4430869d-8103-497c-bbd2-b37433127e64"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_20dfc22a-a89f-4826-b55e-787b04e136df" width="511" height="376" /></a></p> <p>&nbsp;</p> <p>Here I created a Dynamic Column to return the Distinguished Name of the Manager to set against the AD Users.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_a72a5fba-d555-425b-98de-222018c449d1"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_6f01db31-1564-491a-b058-94c62956ffc2" width="622" height="400" /></a></p> <p>&nbsp;</p> <p>Now we just create a simple Schema Map mapping the X_Manager to Manager and using DSID as the Key column.</p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_e26ea8a7-88a6-4c9b-a66a-0cf75464293e"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_29c11434-4984-4954-ae6d-516734f94fa8" width="644" height="419" /></a></p> <p>&nbsp;</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_6920e974-991d-46f8-80d6-06640e7e7419"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_47b3674c-1797-4088-b1d0-4a065ad68f50" width="579" height="137" /></a></p> <p>&nbsp;</p> <p>Now you just need to compare the data and synchronise the changes.</p> <p>&nbsp;</p> <p>You can download and use a fully featured version of Data Sync Studio for 7 Days to see how this AD solution works.&nbsp;</p> <p>&nbsp;</p> <p><a href="http://www.simego.com/Install/Data-Synchronisation-Studio">Click here to go to the Downloads</a></p> Team Blog Thu, 10 May 2012 09:49:45 +0000 http://www.simego.com/Blog/2012/05/Create-an-RSS-Feed info@simego.com (Sean Cleaver) Create an RSS Feed <p>This post is an example of how you can use the Data Sync RSS Provider to create an Exchange Rate RSS feed. You can use a similar approach to create an RSS feed from virtually any kind of data.</p> <p>First we connect Data Source A to the FX Rate provider.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_d9706395-d0fe-451d-bfc4-bca23810cded"><img height="356" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_a83fcd15-afe1-4237-a970-3a6a065a112f" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> <p>We want a couple of additional columns for our RSS File so we create them using Dynamic Columns.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_1a1f415d-2b19-4163-abe3-c39b6799912f"><img height="409" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_73260a7c-9b57-4c65-950b-ed2ae7a2df59" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> <p>We then connect the Target Data source B to the RSS File provider, enter a name for the File and click OK.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_634f0f94-e007-4839-a18b-2f70409bcc1a"><img height="356" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_5ae51835-2661-49a1-8351-66c4e101089c" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> <p>We then create the Map between the FX Rates and the RSS File like this.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_bdfdaa0b-71c9-4b34-94fe-4719483f5475"><img height="399" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_4ecb3600-36fc-49bf-8b66-14a3cd9af42e" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> <p>Schema Map</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_489189e9-bbff-4d6d-b3b4-d2155b7f9312"><img height="145" width="644" src="https://simegoimages.blob.core.windows.net/images/blog_538cb91b-27cc-489b-9074-1b0d93b3f5aa" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> <p>Once you sync up the Data you will have a nice new RSS File that contains the current currency exchange rates from Yahoo.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_80769412-ba4d-4839-aa64-e3f228c58181"><img height="772" width="528" src="https://simegoimages.blob.core.windows.net/images/blog_fd588491-98af-44a1-9f19-e832af66b19a" alt="image" border="0" title="image" style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;" /></a></p> Team Blog Wed, 09 May 2012 16:10:03 +0000 http://www.simego.com/Blog/2012/05/Dynamics-CRM-2011-Import-Contacts-and-Accounts david@simego.com (David Evans) Dynamics CRM 2011 Import Contacts and Accounts <p><strong>Update:</strong></p> <p>Simego DS3 has been updated to allow a much easier drag and Drop import and lookup process, you can easily see in these tutorials:</p> <p>&nbsp;</p> <p><a title="http://www.simego.com/Help/Tutorials" href="http://www.simego.com/Help/Tutorials">http://www.simego.com/Help/Tutorials</a></p> <p>&nbsp;</p> <p>This (no code) post shows how to use Data Sync Studio and Lookup Columns to import Accounts and Contacts from SQL into Dynamics CRM and keep the relationship between Account and Contact.</p> <p>&nbsp;</p> <p>Our SQL Export Database consists of 2 tables “Account” and “Contact” these 2 tables are linked via a Foreign Key Contact.AccountID –&gt; Account.ID.</p> <p>Account SQL Table</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_e370a745-ad33-4b6a-9a39-83c2beea1c1a"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_e5ea00c3-ffc2-4b3a-af7a-2a597f53246c" width="283" height="163"></a></p> <p>Contact SQL Table</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_883679df-1698-4e46-91a5-1d121271af57"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_14d18284-c33f-40c3-8928-fdab4987f9ec" width="644" height="316"></a></p> <p>Dynamics CRM doesn’t work with Integer keys it uses Guids for the Primary Key. In many cases it may be impossible to change the source database to update each record with a Guids. We are using Data Synchronisation Studio which can manage the GUIDs without making any changes to your SQL Database or Dynamics CRM.</p> <h3>Step 1 - Import your Accounts from SQL</h3> <p>This is an easy Data Sync project where you simply import your accounts table into the Accounts entity mapping <strong>name</strong> to <strong>name</strong> and <strong>id</strong> to <strong>accountnumber</strong>.</p> <h4>Project in Data Synchronisation Studio</h4> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_705382a2-2783-4e2d-b166-b101758b3a50"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_27e9e569-3615-4079-a74e-c23700a7cb8f" width="644" height="399"></a></p> <h4>Schema Map</h4> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_5caf9429-7c88-44d6-b6c7-b829f9ecd0fa"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_5c9c42b2-ea09-4ef2-9b49-5c3bd75329fb" width="644" height="147"></a></p> <p>We now just sync the changes into the CRM Account entity.</p> <h3>Step 2 – Import your Contacts</h3> <p>This step requires mapping the Contact table to the Contact entity in CRM mapping the ID value from the SQL Table to the <strong>ExternalUserIdentifier</strong> column in CRM.</p> <h4>Project</h4> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f48e36c1-9888-44c0-b1b5-3bc31b7b8dd0"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_a5b8882a-e30b-4683-9962-5227916d0794" width="644" height="399"></a></p> <h4>SQL Schema Map</h4> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_6653e4c1-605c-4edd-b3ae-c4ab10a8ead7"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_dce93fff-9b6c-49f8-a234-6ed163290343" width="644" height="181"></a></p> <p>We need one more trick to automatically associate the Contact with the imported Account record in Dynamics CRM. For this we define a Lookup between <strong>AccountID</strong> and the <strong>accountnumber</strong> in CRM.</p> <p>Select the <strong>AccountID</strong> column in the Data source window and click the “Add Lookup” button.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_a371fdbd-8b44-4419-8829-062c1e99d519"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_1d40b28e-f776-4009-bb43-858eccb3ea57" width="269" height="247"></a></p> <p>Next configure the Lookup Data source to connect back to the Dynamics CRM account entity.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f097b62b-e019-4284-9c1c-db236d16bd73"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_d1a6c09e-2d19-4291-bdc5-aa25b94804da" width="644" height="356"></a></p> <p>Next define the Lookup relationship by selecting <strong>accountnumber</strong> in the Target Column field.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_54c788aa-2cbe-4878-ae36-63e523adc0dc"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_3c0535ee-a137-45c3-af16-c8956b921e7a" width="393" height="484"></a></p> <p>Now you can use all the columns from the related Account CRM entity in your project.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_b4b00a6a-1b65-4d45-accf-58c4e3994d0b"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_8ecbabea-e149-4cd1-b9e0-74b9eef94dcb" width="271" height="397"></a></p> <p>We now need to map <strong>Lookup1_accountid</strong> to <strong>parentcustomerid</strong> to create the relationship in Dynamics CRM.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_f6e57f4e-506d-4c8e-841c-a4765b21ed79"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_85675fc8-a1c7-4175-8397-82d2e772c344" width="644" height="183"></a></p> <p>and that's it you can sync up the data and see that your contacts are correctly related to your accounts.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_efeedf31-e425-4953-ba29-0c943f0b6560"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_925578bc-85f1-4854-8adf-6335ecef5c3c" width="644" height="369"></a></p> <p>You can download and use a fully featured version of Data Sync Studio for 15 Days to see how this SQL and CRM solution works<span style="background-color: white">.</span></p> <p><a href="http://www.simego.com/Install/Data-Synchronisation-Studio">Click here to go to the Downloads</a></p> Team Blog Wed, 09 May 2012 15:39:06 +0000 http://www.simego.com/Blog/2012/04/Top-Ten-System-Migration-Traps info@simego.com (Sean Cleaver) Top Ten System Migration Traps <p>Considering the impact on delivery dates, testing, training, regulatory compliance and budget, it is always amazing how little focus there is on migration. There are many many traps to fall into when implementing a new system but migration is one stick that the underlying business will beat you with.</p> <p>&nbsp;</p> <p><strong>1. Your project plan has one task that says &lsquo;Migrate Data&rsquo;.</strong></p> <p>This is very easily done as either the team developing the new system or the vendor, can&rsquo;t estimate the effort so it is just left as a single item. Don&rsquo;t do it. Do some analysis that will at least give you a consideration of risk, effort and cost. Without these, managing the expectation of the business will be impossible.</p> <p>&nbsp;</p> <p><strong>2. Not using an automation tool to migrate data.</strong></p> <p>In the past I have conducted migrations based on SQL scripts, batch files, custom code etc. Quickly you realise that there are real benefits to automation. For example the new system has 5 new fields in a table. Rather than going back and redeveloping the migration code, you just point the tool at it and click update. 1 hour saved and no errors introduced, commit it to source control.</p> <p>&nbsp;</p> <p><strong>3. Migrating old data that might better be placed in a reporting system.</strong></p> <p>You should always consider that some data might be better placed in a reporting or document management solution. If you have a 'end of year' balance brought forward into a new system, do you need all of last years transactions or can you just lookup a report showing them?</p> <p>&nbsp;</p> <p><strong>4. You let the vendor leave out migration or worse make it your entire problem.</strong></p> <p>Migrating data is risky and can be expensive; your development team knows how their current system works better than anyone. Make sure you have some migration commitment from the vendor before agreeing the deliverables and costs.</p> <p>&nbsp;</p> <p><strong>5. Assume that the migration with be&nbsp;straightforward because the&nbsp;old system and the new system do the same thing.</strong></p> <p>The architectural requirements of systems mean that they may have completely differing internal structures and security models. These factors influence the effort of migration by factors of 10. Even simple upgrades between versions can be the same as moving to a different&nbsp;vendor&rsquo;s.</p> <p>&nbsp;</p> <p><strong>6. Underestimate the effect of the new security model on migration.</strong></p> <p>Security models are there to make sure that data is not viewed or manipulated by an unauthorised individual. During migrations it is often necessary to circumvent or assume the identity to extract and insert data. The developers of the current systems could make it extremely difficult for you to move data by using encryption, logic in the user screens and even more esoteric (smart at the time) tricks.</p> <p>&nbsp;</p> <p><strong>7. Thinking you can sort out the data quality once it is in the new system.</strong></p> <p>Often a driving reason the move to a new system is that it has better data quality control. For example, we often see telephone numbers saved as &lsquo;TBC&rsquo; in an old system and the new system will only allow number to be saved. A significant phase in migration is cleaning the underlying system so that clean data can be stored easily into the new system.</p> <p>&nbsp;</p> <p><strong>8. Underestimating the involvement from the business to clean the data.</strong></p> <p>As with 7, it is important to realise that users will have to assist in cleaning data. You can make this easier by creating lists of broken records and request rules for your automation tools to follow.</p> <p>&nbsp;</p> <p><strong>9. Not realizing the impact of month/quarter/year ends on the migration.</strong></p> <p>Many accounting systems require journaling and retrospective adjustment. Additionally, when picking a date for migration, make sure the business is available and not buried in month end, quarter end, tax year-end or year end.</p> <p>&nbsp;</p> <p><strong>10. Not documenting the migration process so no one knows where the data came from!</strong></p> <p>When migrating data it is often necessary to transform the data several times before it finally resides in the new system. Often a client will ask, why is the address used and not the billing address from the old system? You need to have a method of tracing the source to target and often you automation tools can help to generate documentation.</p> <p>&nbsp;</p> <p>At Simego we have been involved in assisting clients who have undertaken system migrations.&nbsp;</p> Team Blog Mon, 23 Apr 2012 14:31:26 +0000 http://www.simego.com/Blog/2012/04/Data-Synchronisation-Studio-2012 info@simego.com (Sean Cleaver) Data Synchronisation Studio 2012 <p>The latest version of Data Synchronisation Studio has gone live today, this release has had major re-work to allow loading of much larger data sets and to significantly reduce the memory usage of the application. Our reliance on the .NET DataTable object is now gone and we created a new object to store the data which uses fast disk based storage instead. This also allowed us to to improve Dynamic Columns so that DBNull.Value is now gone and is just NULL and you can now use Nullable&lt;T&gt; in you Dynamic Column properties.</p> <p>For example loading an comparing Data Sets of Millions of rows so now possible. For example below is a screenshot of Data Sync loading and displaying over 18 Million Rows from a CSV file.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_8fcaea68-4d74-459d-abaf-550d6221aa8f"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_08ebe467-6e09-4997-97fb-4ac08f3b6f50" width="644" height="454" /></a></p> <p>We also now have a new Social Panel on the Start Page so that you can see our latest Blogs and Tweets right from within the application.</p> <p><a href="https://simegoimages.blob.core.windows.net/images/blog_054ba718-6805-4600-b41e-db4dd19699f5"><img style="background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;" title="image" border="0" alt="image" src="https://simegoimages.blob.core.windows.net/images/blog_a8725cef-4363-44fe-9688-b3a944438e6e" width="571" height="484" /></a></p> <p>This release includes our new Data Generator Provider which with Dynamic Columns allows for the generation of Sample Test Data or for you to connect to other data sources we haven&rsquo;t created a provider for. With the Test Data Generator you can further transform your data such as creating Pivots etc.</p> Team Blog Mon, 02 Apr 2012 10:15:01 +0000