Things that make it go vroooom……. and not break!

So you’ve installed IBM Cognos Controller, you’ve upgraded the database and you have deployed the client. That in itself is no mean feat but the next question is how to do I make it go faster.

 

Below are a series of enhancements that you can make to your Controller environment that could make a 20% to 30% improvement in general performance.

 

1… Split out the consolidation activities from the main application by creating a second COM+ application.

2… Increase the web server document sizing capabilities in IIS 7.5

3… Use the 64 bit Java Proxy service to improve AFC performance

4… Use LAA (Large Address Aware) to make more memory available for the Controller Client on 64bit Operating Systems.

5… Reduce the amount of time that each Controller system waits for batch jobs.

 

1.. Splitting the Consolidation Activities

IBM Cognos Controller is a single threaded application. Often I hear complaints that users are waiting patiently behind a Consolidation to finish before their reporting jobs running in batch can be delivered. Below are some simple steps to create a second COM+ application called COGNOSVFP which will perform the Consolidation allowing other process to use the existing IBMCOGNOSCONTROLLER COM+ application. It only takes around 15 minutes to complete this work but you MUST get all users out of Controller before you start.

 

Steps:

1. Ensure that no users are using Controller (downtime)

2. Logon to the application server as an administrator

3. Click on ‘Start – Settings – Control Panel – Administrative Tools’

4. Launch ‘Component Services’

5. Expand the ‘top row’ (by clicking on the ‘+’ symbols) until you reach the COM+ application called "IBMCOGNOSCONTROLLER"

6. Expand this, and expand “Roles”

7. Make a note of the name of the group under “Roles” (for example called “ControllerUsers”)

8. Make a note of the users listed underneath (typically they will be “Everyone” and “NT AUTHORITY\system”)

9. Right-click on “IBMCOGNOSCONTROLLER” and choose “properties”

10. Click on tab “Identity”

11. Make a note of the username (for example “mydomain\myserviceaccount”)

12. Ensure that you know the correct password for this user before continuing

13. Close the screen, and right-click on “COM+ Applications” and choose “’New Application”

14. In the wizard, choose ‘Next’ then click ‘Create an Empty Application’

15. By convention, we shall name the application ‘COGNOSVFP’ and ensure ‘server application’ is selected.

16. Choose ‘This user’, and use the same Windows user (also known as the “COM+ user” or "service account") that we located earlier.

17. Click “Next”

18. At the next screen, highlight ‘CreatorOwner’ and choose ‘remove’

19. Then click ‘add role’ and type in the SAME name as the role that we saw earlier (for example ‘ControllerUsers’) then click OK.

20. Click ‘Next’

21. At the next screen, remove any user listed, and then add the entries that we saw earlier in the IBMCOGNOSCONTROLLER role.

22. Click “Next”, “Finish”

23. Expand the IBMCOGNOSCONTROLLER application, and open “Components”. Several hundred individual icons will appear on the right-hand side:

24. Highlight the 2 icons called “frangovfp.cglobvar” and “frangovfp.main

25. Right-click and choose “move

26. In the “Move components” screen, ensure that “COGNOSVFP” is selected, and click “OK”:

27. Expand “COGNOSVFP” – “Components”

28. Select both components, and right-click and choose “properties”

29. Click on ‘Security’ tab, and then tick both boxes ‘Enforce’ and ‘ControllerUsers’

30. Click OK

31. Reboot the application server. You are now ready to test.

 

The COM+ application is shown as the process DLLHOST.EXE within Task Manager. A consolidation would usually take up one entire CPU so on a quad core server I would expect the process to utilise 25% of the total capacity.

 

2.. Increase the Web Server document sizing capabilities

Some protection has been put in place in the versions of IIS shipped with Windows 2008 R2. This means that some large spreadsheets will fail with ‘Out of Memory’ issues or HTTP 404 errors. To overcome this you simply need to edit the file below and larger spreadsheets will run successfully.

 

1. Ensure no users are using Controller (short period of downtime)

2. Logon to the Controller application server as a Windows administrator

3. Launch Windows Explorer

4. Locate your "ControllerProxyServer" folder. TIP: By default, for 64-bit servers this is: "C:\Program Files (x86)\IBM\cognos\c10\ControllerProxyServer"

5. As a precaution, take a copy of the file "Web.config" and place it somewhere safe (as a backup)

6. Click "Start – All Programs – Accessories"

7. Right-click on "Notepad" and choose "Run as Administrator"

8. Click "File – Open"

9. Browse to the location of your "ControllerProxyServer" folder.

10. Change the file format from "Text" files to "All Files (*.*)"

11. Open the file "Web.config" file

12. Scroll all the way to the bottom.

13. Just before the final line ("</configuration>") add a few lines of extra text (see below for details).

14. Save

15. Test.

Additional lines in Italics and Bold

</system.diagnostics> 
<system.webServer>
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="3145728000"/>
</requestFiltering>
</security>
</system.webServer>

</configuration>

3.. Using 64bit Java Proxy Service

This change involves using the 64bit version of Java through the CCR_JAVA_Home variable and changing the executable that the service runs under. This is a new feature that has been released in IBM Cognos Controller 10.1.1 Fix Pack 2 only.

 

1. Ensure no users are using any functionality related to JAVA (or, ideally, ensure no users are in the Controller system at all)

2. Logon to the Controller application server as a Windows administrator

3. Stop the Windows service "IBM Cognos Controller Java Proxy"

4. Launch Windows explorer, and navigate to the …c10\server folder. TIP: by default this is here: D:\Program Files (x86)\IBM\cognos\c10\server

5. Rename the file "ccrproxy.exe" to "ccrproxy.exe.x86"

6. Rename the file "ccrproxy_x64.exe" to "ccrproxy.exe"

7. Right-click on "Computer" and choose "Properties"

8. Click "Advanced system settings"

9. Click "Environment Variables"

10. Highlight the "CCR_JAVA_Home" system environment variable and click "Edit"

11. Modify it to point to the directory of a 64 bit JRE. TIP: By default this is: D:\Program Files (x86)\IBM\cognos\c10\bin64\jre\6.0

12. Start the Windows service "IBM Cognos Controller Java Proxy"

13. Test.

IMPORTANT: It is more than likely that this file will be replaced everytime you patch the Controller software therefore you should ensure that you document this change as part of an update in the future.

 

4.. Use Large Address Aware to allow Controller Clients to use more than 2GB memory

The change below is NOT officially supported by IBM but the people on the ground tell me if works well and I have seen big performance improvements on larger processes with less Out of Memory issues. It is recommended to be implemented on larger PC’s with over 4GB RAM and all Citrix / Terminal Server implementations.

 

This process uses a third party application that is not supported directly by IBM but it is recommended to use it for the performance improvement.

The steps to implement this on each client device are as follows:

1. Open Windows Explorer

2. Navigate to C:\Program Files (x86)\IBM\IBM Cognos Controller.

3. Make a copy of the file ccr.exe

4. Download the third party tool 4GB Patch from http://www.ntcore.com/4gb_patch.php

5. Open the 4GB Patch application.

6. Open the file ccr.exe from the open icon in.

7. The application is changed to allow the Large Addressing and a backup copy it also made.

8. Test the core functionality.

Ensure you do this configuration change on every Citrix / Terminal Server as you will experience slower performance on those servers not patched.

5.. Reduce the Controller Job wait time

Whenever you send a job to the batch server, it can take some time to pickup the request. This can cause frustration as the default is set to 20 seconds per application database. So if you have LIVE, TEST, DEV, and TRAINING databases a consolidation might not start for 80 seconds (4 databases x 20 seconds). My editing the registry you can reduce this time to something more sensible. You could also reduce the number of databases too!!

 

To alter the registry you simply need to perform the following steps on your Controller application server:

1… Open regedit32.exe

2… Navigate to HKEY_CURRENT_USER\Software\VB and VBA Program Settings\FrangoController\Settings

3… Change the BTC_TIMER_INTERVAL from 20000 (20 seconds) to a lower value (minimum 2000). Do not change it to zero.

4… Close the registry editor and restart the IBM Cognos Controller Batch Server service.

 

Please note that the location of the registry key for 64bit Servers is HKEY_CURRENT_USER\Software\Wow6432Node\VB and VBA Program Settings\FrangoController\

Also, it is good practice to check your c10\data directory out and remove any redundant .UDL files in there as they will be scanned by the Batch Service.

Advertisements

Cognos Controller Performance Issues

Overview

I recently completed a review for a client on Cognos Controller performance. The environment was a very simple configuration with only a few users but they were experiencing terrible general performance within Controller itself.

The environment was a single server containing SQL Server and Controller Version 8.5.1 RTM. The user community was only 30 users split over 12 companies and all users use the .Net web client to connect via the Controllerbin virtual directory.

During the review it was decided to look at three key areas:

  1. Logon Performance
  2. Data Entry Performance
  3. Report Execution Performance

We decided to extend the scope of the performance testing by benchmarking operations from the head office and then repeat the tests at 3 of the external sites.

Benchmarking

The logon performance issue appeared to be the biggest issue for the remote userbase. Therefore, we decided to break this test up into two parts. The first one was to time the product from double clicking the URL link for Controller to the moment the actuality screen appeared and then the second test from the actuality screen to Controller’s application being fully loaded.

This test was conducted by ourselves at each site and we also tested each issue using local hardware and the same laptop that we used at Head Office to eliminate hardware differences.

We then repeated the tests for data entry and report generation, again using the same local hardware and group company hardware.

Findings

We found that logon performance shows a vast difference in performance. The statistics shown below are logon time in seconds.

image

image

Clearly, the results show that simple networking is not causing the issue. At site 2 it takes three times longer to logon as it does for a head office user at site 2!

We also tested logon performance at each site using the Head Office laptop using the Web client and the Locally installed client, again this showed further gains in like for like performance.

image

image

This clearly demonstrated the saving that hosting the client locally rather than using the .Net client had on performance.

When we tested Data Entry and Report Generation we saw similar improvement generally in performance.

Data Entry:

image

image

image

image

Report Generation:

image

image

image

image

The hardware used for testing was:

image

Investigation

With the results to hand I started to look at the architecture and the configuration. It was soon clear where the problem resided….. Authentication!!!!

This client had an authentication model based on Cognos8 security rather than the Native security model in Controller. When I investigated this further, not only was active directory used but each site had their own domain and the servers were based in the head office domain.

When the system was originally implemented by another organisation they correctly configured the system to allow for Single Signon and had to setup the authentication to use the multiDomainTrees and chaseReferrals settings. These settings basically search the entire number of domains for authentication and it was this constant searching that was causing performance problems logging on and also running anything as Controller constantly checks that users can perform actions and see data.

During our investigation, we also found that Enhanced Report Optimisation (ERO) was not implemented. This is known to improve Excel report performance by anything up to 300%.

Recommendations Implemented

The following recommendations were implemented:

  1. Changed Controllers security model to ‘Native’
  2. Implemented Enhanced Report Optimisation.

Post implementation testing

Following the configuration changes we repeated the benchmark testing and found significant improvements to the performance of all areas.

The collates results showed improvements in the following areas:

  1. Logon Performance – 85% improvement.
  2. Data Entry Performance – 45% improvement.
  3. Report Generation – 25% improvement.

Conclusions

  1. If you have Cognos8 security and multiple domains then you should expect application delays generally whilst the CAM module performs authentication.
  2. Use Native security if you have a small number of users to manage.
  3. If you have a multi-domain environment, consider deployment of Controller via Citrix or Terminal Services to reduce management and improve performance.
  4. Always implement Enhanced Report Optimisation whenever you install Controller. It will pay benefits immediately.

I have worked with IBM on the results of this engagement and the following Technote has been created with a brief summary at http://www-01.ibm.com/support/docview.wss?uid=swg21606662

I also recommend that before you implement Cognos8 security with Controller that you consider reading the Active Directory Story at http://www.ibm.com/developerworks/data/library/cognos/page64.html

Cognos Controller 10.1.1 is released

IBM Cognos Controller 10.1.1 has recently been released and contains the below exciting new features:

  • Copy Journals – Keep creator
    When you copy a journal in IBM Cognos Controller 10.1.1, then the user that originally created the journal in data entry is kept as the creator and is not replaced by the user that did the copy.
  • Improved company status handling with company journals
    In IBM Cognos Controller 10.1.1 the status handling of companies has been improved.
  • Improved delete period values
    In IBM Cognos Controller 10.1.1 the deletion of data has been improved.
  • Concurrent updates of structures
    IBM Cognos Controller 10.1.1 no longer has any restrictions on what windows can be open simultaneously.
  • Improved status functionality
    In IBM Cognos Controller 10.1.1 the status functionality has been improved by introducing two new command centers.

A significant number of bug fixes have also been applied and a few improvements to the installation process including no need for the Windows 2003 Support Tools (hurrah!).

The full list of bug fixed in this release can be found at http://www-304.ibm.com/support/docview.wss?uid=swg27024070 and I strongly recommend that you check this list out as some of the fixes are quite significant.

The supported environments have been updated to include Internet Explorer 9 and Adobe Reader X. However, support for SQL Server 2012 is not yet there! Full details can be found at http://www-304.ibm.com/support/docview.wss?uid=swg27024069

 

Technical Installation Changes

JRE

From a technical perspective the Java Runtime Environment (JRE) is automatically installed with IBM Cognos Controller Financial Analytics Publisher and IBM Cognos Controller Web Services Server. This means that you no longer need to define the JAVA_HOME or CCR_JAVA_HOME environments.

You only need to define these environments if you want to use an additional JRE in a different location.

Financial Analytics Publisher – FAP

The IBM Cognos Controller Financial Analytics Publisher (FAP) installation was divided into two; FAP Server installation and FAP Client installation.

This simplifies the installation of the FAP server and FAP client on different machines.

Additionally, the FAP service is automatically registered after you install the FAP Server. By default, the FAP service is registered to use the local system account. However, you can change this account to suit your requirements.

Application of Updater Kits

IBM Cognos services are automatically stopped when you install interim fixes or fix packs.

The updater kit automatically checks for IBM Cognos services that are running on your machine, and attempts to stop them. The updater kit displays a list of services that are currently running. If any of the services cannot be automatically stopped, the user is prompted to manually stop them. The updater kit continues with the installation after all services are successfully stopped. This issue has previously caused lots of issues when applying fix packs or interim fixes.

 

As well as these improvements, some additional changes have been made to existing functionality, namely:

Unpacked integration directory for CCRLocalClient.msi

Administrators use the CCRLocalClient.msi when full control over the deployment process is required. Now the integration folder is unpacked directly in the CCRLocalClient.msi file.

In previous versions, the CCRLocalClient.msi contained an integration.zip file that was unpacked in each user’s profile at the launch of ccr.exe. Now that the integration folder is unpacked directly in the CCRLocalClient.msi file, there is only one unpacked instance of the integration folder. This improves the speed and efficiency of client operation on all client installations especially those deploying over Citrix or Terminal Services.

Microsoft Windows 2003 support tools are not required

IBM Cognos Controller no longer depends on the inclusion of Microsoft Windows 2003 support tools. Controller used to require a single DLL from this software but finally this is no longer the case.

Configuration settings preserved by updater kit

Configurations settings are no longer over written when you install interim fixes or fix packs for IBM Cognos Controller.

Now, when you install an interim fix or fix pack using an updater kit, a set of default configuration files are included. This allows you to keep the configuration settings that you used prior to installing Controller updates. These default configuration files are easily located since they all use the file extension .new.

If you want to replace any of your existing configuration files with a .new default configuration files, you simply remove .new from the filename. For example, to use the updated version of the ccr.exe.config.new file rename the file as ccr.exe.config.

The default configuration files provided with the updater kit are:

  • web.config.new, located in c10_location\ControllerProxyServer
  • import.config.new, located in c10_location\ControllerProxyServer
  • CCR.exe.config.new, located in c10_location\webcontent\ccr\
  • FAPService.properties.new, located in c10_location\server\FAP

Improved handling of the Windows User Access Control

To correctly configure all settings during the installation and configuration or to install the local client, you must perform these tasks with elevated privileges as an administrator.

If you are installing on a Windows Vista, Windows 7, or Windows 2008 computer, the User Account Control (UAC) is turned on by default. When the UAC is turned on, it can suppress certain programs from accessing system files or registry keys without full administrative rights. You must run the following IBM Cognos Controller programs under administrative privileges:

IBM Cognos Controller Configuration
Adjust the permission level of your user account to correctly store changes to the configuration settings.
CCRLocalClient.msi package
Adjust the permission level of your user account to properly install the local client.

Windows informs you that a change made to your computer or installing a new program requires administrator-level permission. As an administrator, you can acknowledge the notice and continue. If you are not an administrator, or if you are using a standard user account, you must provide either permission or an administrator password before performing the task. With the proper handling of UAC, changes to the configuration are accurately stored, reducing the likelihood of any associated installation errors.

Setting the level of detail in error messages

Administrators can now specify the level of details that are displayed in an error message to help users troubleshoot a problem. The administrator can change this setting by modifying the ControllerWebServiceException property in the Web.config file.

Error messages include details that can help IBM Support diagnose the problems and trace the events that led up to the failure. As a system administrator, you can now restrict the type and quantity of information that is displayed in the error message as a security precaution. For example, when the property is set to 0, no details are provided except for the instruction to contact the system administrator.

To facilitate the configuration process, the error message setting also applies to those error messages generated when using IBM Cognos Controller Configuration.

Summary

So in summary, IBM have been busy in the R&D offices putting together a strong release of Controller with particular emphasis on listening to the Customer and Partner communities feedback for new functionality, ease of installation and operation improvements. This release also works in conjunction with the other 10.1.1 family of products. Personally, I have linked Controller 10.1.1 to BI 10.1.1 FP1 and TM1 10.1 FP1 without any major issues.

Controller will continue to be a challenge to install and configure because that is the nature of the beast especially on non SQL Server environments but it is a step in the right direction.

As usual, I would love to hear your experiences with 10.1.1 and also 10.1. We have been working with it for a while and are happy to share experiences with the community.

Controller and TM1 Integration

Due to the recent release of TM1 Version 10.1, a number of clients are wanting to utilise the latest functionality in TM1 Version 10 with the Financial Analytics Publisher (FAP) functionality.

This can now be done but you need to observe the following matrix of compatibility carefully:

  • Controller 8.5.0 only works with TM1 Version 9.4 and 9.5.1
  • Controller 8.5.1 only works with TM1 Version 9.5.1 and 9.5.2
  • Controller 10.1 RTM to 10.1 Interim Fix 5 works with TM1 Version 9.5.2
  • Controller 10.1 Interim Fix 6 works with TM1 9.5.2 and TM1 10.1

Controller 10.1 IF6 was released on March 7 and is the only release that integrates TM1 10.1 with Controller and the FAP process. Additionally, to allow the TM1 10.1 integration you need to perform the following ‘manual’ step.

  1. Replace the “c8itk4j.jar” from the “FAP lib” folder with the “c8itk4j.jar” file from the “TM1_10.1_CAM_compatibility” folder in Controller 10.1 IF6.
  2. Confirm that the correct system variable Path is set for the newly installed TM1.
  3. Restart the FAP service.

Once you complete these steps you break any integration with TM1 9.5.2 so backing up the existing c8itk4j.jar file before you start is highly recommended.

It is also planned that this step will be automatic when patch updating Controller  from release 10.1 IF7 (Due April 2012) and this means that from IF7 onwards FAP integration will only be provided by using TM1 10.1.

The Interim fixes that are produced by the Controller development team are only made available on request. If you wish to obtain an interim fix then you need to must log a PMR with IBM Support. The support team usually send you a link to the FTP site within 24 hours to entitled support customers. If you are supported by a partner then you need to request the Interim Fixes through your partner support desk.

It is interesting to see IBM make the bold step of providing integration to the newest platform only so early in it’s lifecycle. It shows the confidence that they have in the new version of TM1.