Some of our more cutting-edge clients (thanks Dave!) asked how we might help them leverage Cloud Computing to host their applications. The advantages of cloud computing are easy to see because the cost of storage and computing is measured in pennies per hour/gigabyte and works especially well for reporting and analysis applications.

Here are some examples (a la Mastercard style!)

Applications that have archives that are rarely used but have sophisticated, heavy duty backup machines to support them. Instead, move them to the cloud where they are restored for temporary use when needed

$ .10 (Storage costs per GB per month)

Development team needs 8 weeks lead time from their data center to setup new development/test or production environments. Instead, start as many servers as you need in minutes on the cloud

$ .45 (per hour of server uptime)

Applications that run the monthly close from a workstation, *cough* I mean, server, that sits under the CFOs desk. Instead, move them to the cloud with built in reliability and disaster recover readiness

$ 334 (Server usage cost per month)

Applications that only get used heavily during the budgeting process, using large servers but stay idle the rest of the year. Instead, move it to the cloud and keep the machine up only for 2 months of the budget cycle

$ 669 (Server usage per month)

Peace of mind, reduced expense and no capital outlay

Priceless

These economics are fundamentally changing the way we build all kinds of applications, let alone datawarehousing, reporting and analytics. And don't let the prices fool you, the CPU and memory of these on demand virtual instances are on par with intel machines in any data center.

In addition to the above use cases, simpler examples include quick proof of concepts, development and test environments. But why stop there? if it is working, why not migrate it to production as well?. But sticky issues around enterprise software licensing in the clouds remain, so we were pleasantly surprised by Oracle's announcement that they are supporting the licensing in the cloud. Its still not a true metered licensing that would work seamlessly for a cloud platform, but it's a start.

Security is also a concern, but cloud computing providers such as Amazon and Google are makingsignificant efforts to secure the infrastructure (Amazon's security whitepaper). According to Techcrunch, a goodchunk of Amazon web service customers are banks and pharmaceutical firms.

So, knowing this was an idea whose time had come - we stuck our heads in the clouds (sadpun intended) and used our expertise in implementing, scaling, and managing BI applications to layer Oracle EPM on top of Amazon'scloud computing service EC2. Other BI stacks like HyperRoll will follow soon. Our efforts have resulted in a fullymanaged, top to bottom platform that includes the OS, datawarehouse,OLAP reportinginfrastructure and is now ready for your application. This is the cubeFlex platform, powered by Amazon Web Services! 

To learn more about cubeFlex, I would like to invite you to apply for access to one of the Oracle EPM beta instances by signing up here. The instance has a full stack of Oracle EPM software ready to be configured. Apologies in advance, because interest has been high we may have to put you on awaiting list.

This post demonstrates the use of Amazon S3 to archive and restore Oracle Essbase Cubes.

The advantages of doing this include: 

• Virtually unlimited storage at pennies per GB per month. 
• Data geographically and asychronously stored
• You only pay for what you use

The usual prerequisites apply:

• Get an Amazon AWS account here
• Followed these instructions and have your AWS Access Key, Secret AWS Access Key and "pem" files handy.

Set the environment up

The other prerequisites, Ruby (RPM) and S3Sync are already installed and ready to go on the cubeFlex AMI. You do have to set three environment variables, this can also be set in the ~/.aws_env or manually:

[full360@domU]# AWS_ACCESS_KEY_ID=xxxx[full360@domU]# export AWS_ACCESS_KEY_ID[full360@domU]# AWS_SECRET_ACCESS_KEY=xxxx[full360@domU]# export AWS_SECRET_ACCESS_KEY[full360@domU]# export S3_ARCHIVE_BUCKET=archive.full360.com Backup the cube

Now you can use the cubeFlex to backup any of your Essbase applications to S3, this script will do a level zero export of the database, copy the outline, package the two files and upload to  backup the essbase outline file and level zero data on S3:

[full360@domU]# ./f360/scripts/ec2_hyp_app_backup.sh XYZ XYZ

You should have your backup file XYZ.tgz on your specified S3 location, in this case archive.full360.com

Restore the cube

To restore your application back from S3 Execute the command:

[full360@domU]# /f360/scripts/ec2_hyp_app_restore.sh \<Application Name> <Database Name> <ASO/BSO>

This script downloads the packaged zero level export and outline file from S3 and restores the application back on Essbase server. For Example

[full360@domU]# ./f360/scripts/ec2_hyp_app_backup.sh XYZ XYZ ASO

You should have your application XYZ back on your Essbase server, with data and default aggregations ready. Log files for the process can be found in /f360/scripts/logs

 

Please find the below steps and screen shots to configure Hyperion System9 products

Start configuration tools from .../f360/hyperion/common/config/sh configtool.sh -console



The screen wil display the list of products you installed and pending for configuration, select the pending products by keying respective numbers and then press 0 to continue...



Select all the sub products which are listed for pending



Select database oracle



Enter Oracle database login details for system9 repository.



Select the application server Apache Tomcat



Conitune to select default SharedServices Name and Port



Continue to complete Shared Services configuration



Once the Shared Services configuration is succeeded, start the shared services from the below location
../f360/hyperion/deployments/Tomcat5/bin/sh startSharedServices9.sh

Continue to select other products and enter the user id admin and password as password to resgister with Shared Services



Keyin the database details and select the default ports for reporting and analysis configuration below











Key in the below deatils for Provider services and Essbase server





 

In case of any queries, please free to drop a note on full360 support.

Oracle has now officially certified 11g, Oracle XE, Enterprise Linux and Oracle Fusion for Amazon Web Services' Elastic Computing Cloud (AWS EC2). Information about AWS and EC2 can be found here.

In essence, Amazon is best known for its e-commerce site, but a few years ago, a team within Amazon began to look at a way to leverage the excess capacity they were seeing in their data centers. Eventually this grew into AWS. What Amazon is providing as a service is the ability to utilize its network, hardware and systems management expertise as a service. This allows us to request virtual pieces of this infrastructure on demand and scale these virtual instances almost without limit.

What is amazing is that you only pay for the compute time and space that you use. These virtual instances are based on templates called Amazon Machine Images (AMI). Oracle has provided free, prebuilt AMIs for the certified products.

They have not yet released Oracle Essbase for the cloud, the following steps show how we built our AMIs for Essbase. We have done the hard work for you and here is a blog post describing how to use it. If you are one of those brave souls that likes to tinker, here's how we did it.

I will not be describing how to setup your Amazon credentials and certificates that you need to start/run/use these images. There are a bunch of articles out there describe the steps. Here is one, and here's another - this includes a lengthy how-to on installing XE.

We went thru the process of building the image from scratch using VMWare. Since Oracle Enterprise Linux 4.0 (not 5 which is what all the oracle AMIs are based on) is what Essbase 9.3.1 is supported on, that is the version we are going to install. It can be downloaded from here and installed on the VMWare image using the minimal configuration (for a small tight image).

Then installed the following (rpm -Uvh <rpm file>):

Install jre (http://java.sun.com/javase/downloads/index_jdk5.jsp)Install curlInstall libaioInstall libidnInstall rubyInstall smart (makes linux soooo much easier to manage) and configure it to use the local rpmsConfigure Smart: $ smart channel --add local-EL-rpms name="Oracle Enterprise 4 Linux RPMs" \manual=true type=rpm-dir path=/mnt/oel4-rpms/ -y$ smart channel --add Dag-Wieers type=rpm-md priority=-5 \baseurl=http://apt.sw.be/redhat/el4/en/i386/dag/ -y$ smart channel --add VA-Tech-Extras type=rpm-md priority=-5 \baseurl=http://mirror.cs.vt.edu/pub/CentOS/4.4/extras/i386/ -y$ smart channel --add VA-Tech-Base type=rpm-md priority=-5 \baseurl=http://mirror.cs.vt.edu/pub/CentOS/4.4/os/i386/ -y$ smart channel --add Dries type=rpm-md priority=-5 \baseurl=http://ftp.belnet.be/packages/dries.ulyssis.org/redhat/el4/en/i386/dries/RPMS/ -y$ smart channel --add atrpms name="ATrpms Repository" type=rpm-md \priority=-5 baseurl=http://dl.atrpms.net/el4-i386/atrpms/stable/ -y

(AWS virtualization doesn't like tls)

$ mv /lib/tls /lib/tls.disabled

We use s3sync to transfer data to and from s3, so we installed that. S3Sync can be found here

Unfortunately, even the small minimal install starts a bunch of services that are not needed on a cloud based instance, so We turned off the following services (http://fedoranews.org/mediawiki/index.php/Which_Services_Can_I_Disable%3F)

$ chkconfig --level 0123456 nfslock off$ chkconfig --level 0123456 nfs off$ chkconfig --level 0123456 portmap off$ chkconfig --level 0123456 pcmcia off$ chkconfig --level 0123456 smartd off$ chkconfig --level 0123456 rpcidmapd off$ chkconfig --level 0123456 cups off$ chkconfig --level 0123456 gpm off$ chkconfig --level 0123456 isdn off$ chkconfig --level 0123456 rhnsd off$ chkconfig --level 0123456 rpcgssd off$ chkconfig --level 0123456 apmd off$ chkconfig --level 0123456 openibd off

Since AWS is based on Xen virtualization, a few tweaks to the VMWare image are necessary:

$ wget http://s3.amazonaws.com/ec2-downloads/modules-2.6.16-ec2.tgz$ tar -zxvf modules-2.6.16-ec2.tgz -C /$ moddep -a

/etc/sysconfig/network-scripts/ifcfg-eth0 should read:

DEVICE=eth0BOOTPROTO=dhcpONBOOT=yesTYPE=EthernetUSERCTL=yesPEERDNS=yesIPV6INIT=no

After that the ec2 tools are downloaded, and installed in /usr/local/bin

$ wget http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip$ wget http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip

Next is to download Oracle XE from oracle; Download the installations for Essbase, Shared Services and whatever else you need for Essbase from edelivery.oracle.com. We uploaded all the rpm's, Essbase Installations to s3

With the certificates and tools setup, the VMware image is bundled and uploaded to S3, and then registered

$ export EC2_PRIVATE_KEY=/mnt/.ec2/pk-XXXX.pem$ export EC2_CERT=/mnt/.ec2/cert-XXXX.pem$ ec2-bundle-vol -d /mnt -k $EC2_PRIVATE_KEY -u XXXXX -c $EC2_CERT -r \ i386 --no-inherit -p oel4-base-v08$ ec2-upload-bundle -b oel4-base-v08 -m /mnt/oel4-base-v08.manifest.xml -a XXXX -s XXXX$ ec2-register oel4-base-v08/oel4-base-v08.manifest.xml

After starting an instance of the image, download all the installations and rpms from s3 that we uploaded earlier to /mnt using S3Sync

a few more installs using smart

$ smart install compat-libstdc++-33-3.2.3-47.3@i386$ smart install xauth$ smart install xfstools$ smart install oracle-xe$ /etc/init.d/oracle-xe configure

We then installed Shared Services, tweaked it a bit for the lack of tls, installed Essbase and the rest of the gang. Then we rebundled and uploaded the image to s3. That results in our ami!

I know there are a few details I have glossed over, but if you get stuck post a comment in our support forum.

Resources

http://www.stevekallestad.com/wiki/Setting_up_an_EC2_instance_using_VMWare
http://craiccomputing.blogspot.com/2007/01/configuring-oracle-database-xe-on-linux.html
Amazon EC2 Technical Documentation
Using s3sync with EC2

Installing Xming
Xming is a Windows application. You can download the "Xming Microsoft Windows program installer" using the below link http://www.straightrunning.com/XmingNotes/ it’s a normal Windows executable setup.exe. So, run it and follow the on-screen instructions.

Installing PUTTY
Most likely you will want to use ssh to connect to the remote host. So install PuTTY as a ssh client for Windows: http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html

Starting Xming
When you look in the Windows Start Menu you'll find icons for Xming and XLaunch. XLaunch lets you select from the available configuration options and start the Xming server. I don't plan to describe it, since it is really just a launcher for the Xming server and I like the free floating native windows on my Windows desktop.

• Double click on the Xming icon to start the X window system.

(You won’t see anything happen besides a small X icon in the system tray)

• Choose any of the saved Putty sessions to connect to a computer and click on Load

(example: “AWS-Instance1” will load the settings for “ec2-67-202-34-71.compute-1.amazonaws.com”



• If you are using a low bandwidth link then you can enable compression in the Connection->SSH category.
  
  
  
  
• Go To Connection->SSH->X11 settings on putty window, enable X11 forwarding and set X display location to localhost:0





• Click Open to connect to server





• Set display option to localhost:0 on SSH console





• Test any commands which requires X windows, as below.

I am going to show you how to get started with Essbase using our Amazon Images. Using the AMI, I am able to have a fully installed EPM machine with Shared Services, Essbase, Administration Services, Integration Services, Provider Services, BI Services and UI Services in 15 mins. Not that you would EVER want to run all of those on the same machine! So lets get started

Prerequisites

• Get an Amazon AWS account here
• Followed these instructions and have your AWS Access Key, Secret AWS Access Key and "pem" files handy.
• Installed Putty (download here). I recommend using the Windows installer to install all the putty components. Or use a mac - :) X11 and ssh are built in
• Install Xming from here (optional, you will need this if you want to use the GUI config utility)
• You may have to install Java, get it here
• Sign up for the Basic Edition of Cube Flex here (limited number of invitations available) or sign up for the Professional or Enterprise Edition and you wont need the rest of this guide :)

Getting Started

Note: The following set of instructions assumes that you are using Windows. If you are using a Mac, the instructions are similar, except you dont have to set up Putty, just use the built in ssh client.

Download the Amazon Tools from here. Unzip and copy the folder to "C:\Program Files". Rename the folder to ec2-api-tools.

Run the following in a command window to set the environment up, Generate security keys and start an instance of the Full 360 Essbase/EPM AMI.

set PATH=C:\Program Files\ec2-api-tools\bin;%PATH% set JAVA_HOME=C:\Program Files\Java\jre1.6.0_07 C:\>set EC2_CERT=c:\ec2\cert-XXXXXX.pem C:\>set EC2_PRIVATE_KEY=c:\ec2\pk-XXXXXX.pem # Setup the keys C:\cd ec2 C:\>ec2-add-keypair.cmd f360-keypair > "C:\ec2\f360-keypair" # List the Full 360 images (our amazon owner id is 073631148609 C:\>ec2-describe-images -o 073631148609 IMAGE   ami-2fdc3846    cubeflex.full360.com/pub/essbase-32bit/oracle-epm-9.3.1.v01.manifest.xml        073631148609    available       public  9A5AAF61 i386    machine # We are now ready to start an instance # using the AMI id returned by the above command (ami-xxxxx) C:\ec2>ec2-run-instances ami-2edc3847 -k f360-keypair RESERVATION r-14d1037d 073631148609 default INSTANCE i-c845e6a1 ami-2edc3847 pending f360-keypair 0 m1.small 2008-09-29T01:30:13+0000 us-east-1b 

If you see this error : Client.AuthFailure: Subscription to ProductCode 9A5AAF61 required.
That indicates that you have not subscribed to the AMI via amazon. Please sign up here

We can now query the status of the machine start up, if everything is looking good, the result will show that the INSTANCE is running:

C:\ec2>ec2-describe-instances RESERVATION r-14d1037d 073631148609 default INSTANCE i-c845e6a1 ami-2edc3847 ec2-67-202-63-237.compute-1.amazonaws.com domU-12-31-38-00-9D-E1.compute-1.internal running f360-keypair 0 m1.small 2008-09-29T01:30:13+0000 us-east-1b

Woohoo! We are now ready to connect and use the instance! But before that we need to open a port thru Amazon's security to allow us to connect via Putty/SSH.

C:\ec2>ec2-authorize default -p 22 GROUP default PERMISSION default ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0

 

Copy the f360-keypair to f360-keypair-putty, open the file in notepad and delete the first line that starts with KEYPAIR, save the file

Open c:\ec2\ in windows explorer

Right click on the f360-keypair file, and select open. Choose Puttygen from the Putty installation in C:\Program Files

You should see the following dialog box, Click ok

Click on Save Private Key, this saves a Putty format of the private key

Lets connect to the machine!

Open Putty, and in Session > host name field set the hostname from ec2-describe instances - it would be something like ec2-XX-XXX-XX-XXX.compute-1.amazonaws.com:

In the Connection > SSH >Auth menu, choose the ppk file that we just saved:

In the Connection > Data section set the username to be full360

Now go back to the Session Menu, type in cubeFlex in the Saved Sessions box, and click Save:

Now Click Open, which should connect you to your instance.

If you are successfully connected, we need to upload YOUR certificate files (currently in c:\ec2) up to the instance, so go back to the NT command window and run the following. It uses the saved putty session (cubeFlex) to establish the connection:

C:\ec2>"c:\Program Files\PuTTY\psftp" -load cubeFlex Using username "full360". Remote working directory is /home/full360 psftp> cd .ec2 Remote directory is now /home/full360/.ec2 psftp> lcd c:\ec2 New local directory is C:\ec2 psftp> put cert.pem local:cert.pem => remote:/home/full360/.ec2/cert.pem psftp> put pk.pem local:pk.pem => remote:/home/full360/.ec2/pk.pem psftp> quit

We are almost done! Go back to the putty session so we can run setup.sh, you can open this file and take a look, by default it will install Shared Services and Essbase, if you'd like to install anything else, just turn on the appropriate flag

[full360@domU-xx-xx-xx-xx-xx-xx ~]$ cp setup.sh.full360 setup.sh [full360@domU-xx-xx-xx-xx-xx-xx ~]$ ./setup.sh Thats it! Essbase and Shared Services are installed! To configure the installation [full360@domU-xx-xx-xx-xx-xx-xx ~]$ su - oracle -bash-3.00$ cd /f360/hyperion/common/config -bash-3.00$ ./configtool.sh -console

it change user to oracle and go to the hyperion home directory and run configtool.sh -console If you'd like to use the java gui, Naren written a post on how to connect Xming to an EC2 instance

To shutdown the machine:

C:\ec2>ec2-terminate-instances i-c845e6a1 Appendix: Setup Log
creating and attaching volume instance ID is i-c845e6a1 availability zone is us-east-1b creating new volume sized 5 GB Status of vol-610ce908: available ec2attvol vol-610ce908 -i i-c845e6a1 -d /dev/sdi ATTACHMENT vol-610ce908 i-c845e6a1 /dev/sdi attaching 2008-09-29T02:13:55+0000 Status of vol-610ce908 : attached meta-data=/dev/sdi isize=256 agcount=8, agsize=163840 blks = sectsz=512 data = bsize=4096 blocks=1310720, imaxpct=25 = sunit=0 swidth=0 blks, unwritten=1 naming =version 2 bsize=4096 log =internal log bsize=4096 blocks=2560, version=1 = sectsz=512 sunit=0 blks realtime =none extsz=65536 blocks=0, rtextents=0 creating and attaching volume instance ID is i-c845e6a1 availability zone is us-east-1b creating new volume sized 10 GB Status of vol-600ce909: available ec2attvol vol-600ce909 -i i-c845e6a1 -d /dev/sdh ATTACHMENT vol-600ce909 i-c845e6a1 /dev/sdh attaching 2008-09-29T02:14:33+0000 Status of vol-600ce909 : attached meta-data=/dev/sdh isize=256 agcount=16, agsize=163840 blks = sectsz=512 data = bsize=4096 blocks=2621440, imaxpct=25 = sunit=0 swidth=0 blks, unwritten=1 naming =version 2 bsize=4096 log =internal log bsize=4096 blocks=2560, version=1 = sectsz=512 sunit=0 blks realtime =none extsz=65536 blocks=0, rtextents=0 Enabling Oracle Fixing old hostname domU-12-31-38-00-A4-A2 for Oracle Ora Files in /usr/lib/oracle/xe/app/oracle/product/10.2.0/server with correct hostname domU-12-31-38-00-9D-E1 Starting Oracle Net Listener. Starting Oracle Database 10g Express Edition Instance. LSNRCTL for Linux: Version 10.2.0.1.0 - Production on 28-SEP-2008 22:14:56 Copyright (c) 1991, 2005, Oracle. All rights reserved. Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC_FOR_XE))) STATUS of the LISTENER ------------------------ Alias LISTENER Version TNSLSNR for Linux: Version 10.2.0.1.0 - Production Start Date 28-SEP-2008 22:14:43 Uptime 0 days 0 hr. 0 min. 13 sec Trace Level off Security ON: Local OS Authentication SNMP OFF Default Service XE Listener Parameter File /usr/lib/oracle/xe/app/oracle/product/10.2.0/server/network/admin/listener.ora Listener Log File /usr/lib/oracle/xe/app/oracle/product/10.2.0/server/network/log/listener.log Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC_FOR_XE))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=domU-12-31-38-00-9D-E1.compute-1.internal)(PORT=1521))) Services Summary... Service "PLSExtProc" has 1 instance(s). Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service... Service "XE" has 1 instance(s). Instance "XE", status READY, has 1 handler(s) for this service... Service "XEXDB" has 1 instance(s). Instance "XE", status READY, has 1 handler(s) for this service... Service "XE_XPT" has 1 instance(s). Instance "XE", status READY, has 1 handler(s) for this service... The command completed successfully Installing SharedServices Resolving s3.amazonaws.com... 72.21.211.161 Connecting to s3.amazonaws.com|72.21.211.161|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 248,386,185 (237M) [application/x-gzip] 100%[====================================>] 248,386,185 10.06M/s ETA 00:00 22:15:33 (6.55 MB/s) - `/f360/installs/oracle_epm/9.3.1/SharedServices.tgz' saved [248386185/248386185] SharedServices/ SharedServices/setup.jar SharedServices/setup.bin SharedServices/setup.sp /bin/su -s /bin/bash oracle -c /mnt/SharedServices/setup.bin -options /mnt/init/response/SharedServices_install.txt -silent Setting firewall configuration for Shared Services Successfully Installed SharedServices applying LDAP patch Successfully applied LDAP patch Installing ESSBASE Resolving s3.amazonaws.com... 72.21.207.193 Connecting to s3.amazonaws.com|72.21.207.193|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 193,609,216 (185M) [application/x-gzip] 100%[====================================>] 193,609,216 10.14M/s ETA 00:00 22:16:59 (9.90 MB/s) - `/f360/installs/oracle_epm/9.3.1/ESSBASE.tgz' saved [193609216/193609216] ESSBASE/ ESSBASE/setup.jar ESSBASE/setup.bin ESSBASE/media.inf /bin/su -s /bin/bash oracle -c /mnt/ESSBASE/setup.bin -options /mnt/init/response/ESSBASE_install.txt -silent Setting firewall configuration for ESSBASE Successfully Installed ESSBASE Not Installing EPS Not Installing EAS Not Installing EIS Not Installing BIS Not Installing UIS save Firewall config Setup Script Done!

Oracle released its AMIs for the cloud. The important piece of news in this is that customers can use their oracle licenses on the cloud, with the following conversion:

"When licensing Oracle programs with Standard Edition One or Standard Edition in the product name, the pricing is based on the size of the EC2 instances.  EC2 instances with 4 or less virtual cores are counted as 1 socket, which is considered equivalent to a processor license. For EC2 instances with more than 4 virtual cores, every 4 virtual cores used (rounded up to the closest multiple of 4) equate to a licensing requirement of 1 socket."    Not a true cloud based licensing strategy, because it doesn't provide customers the ability to scale and reduce scale on demand. Something is better than nothing though!   It doesn't cover any of the BI products like Essbase, but we have a solution for that ;)   

Here is the oracle website page for the cloud - link