Archive for the ‘General’ Category

Big Data in the Cloud

December 25, 2012

It is hard to say which of the two buzz words tops IT news today: Big Data or Cloud? It is much easier to see how those can play nicely together.

Big Data symbolizes the explosion of computer generated data which is said to double every year outgrowing the capacity of IT data centers. Trying to cope with ever increasing data volumes and make sense of the data companies find themselves in a desperate need for high scale data aggregation, processing and analysis tools.

I won’t dare giving another definition to such a multi faceted thing as Cloud. Let me rather summarize a few of its inherent capabilities representing different levels of the cloud stack:

– On demand resource provisioning

This is a key attribute of the IaaS clouds where you can request CPU, memory and storage resources based on the application demand and release them as they become no longer needed. This is by far the most important benefit IT professionals realize from the Cloud today.

– Scalable data processing

Thanks to the bloom of open source technologies like MapReduce and their various commercial implementations scalable data processing becomes a part of the cloud development platform or PaaS.  It lets application vendors harness the power of infinite cloud resources to perform complex computational tasks on the data.

– Rich data analytics services

The underlying cloud infrastructure enabled a plethora of cloud services collectively known as SaaS that use on demand resources and scalable data processing algorithms to exploit domain specific knowledge and provide actionable insight into data of a different kind.

Looking at all these features of the cloud it becomes apparent that it has a lot to do with big data and that the latter can be well managed in the cloud. Here are just a few examples that show how big data problems can be effectively solved in the cloud.

1. Cloud based data migration services is a natural fit especially if migrated data itself finds a new home in the cloud.

OnDemand Migration for Email is a cloud based service from Quest Software that automates migration of large on-premise email workloads such as Microsoft Exchange mailboxes to the cloud based email systems like Microsoft Office 365. The service relies on the elasticity of Microsoft Azure to provide unique benefits to its customers such as predictable project deadlines, controlled costs and ease of migration. Since the migrated data ultimately settles in a secure cloud email system it largely alleviates concerns about security of data trusted to the cloud service for the time of migration.

2. Application performance monitoring services like Quest Software’s Project Lucy exploit multi tenant nature of the cloud based services to define the “golden standard” of application performance and pinpoint performance degradation long before it adversely affect its users. Project Lucy correlates application performance metrics and configuration snapshots collected from its entire customer base – something that would never be possible for application performance monitoring solutions isolated within a customer’s own data center. Cloud skeptics are left with nothing to be concerned about. No personal identifying data leaves organizational boundaries and only averaged out application performance metrics and anonymous configuration options get sent to the cloud.

3. SIEM solutions like Dell SecureWorks also find a good use of the cloud technology for the threat monitoring use case. SIEM need to reduce overwhelming amounts of logs generated by applications, systems and network devices to detect and respond to security threats. There are two ways to do that: monitoring patterns of known malicious activity and continuous evaluation of user behavior profiles aka advanced persistent threats. Both tasks are very resources intensive and subject to a lot of false positive conclusions. Cloud based SIEM can leverage dynamic resource provisioning and cross customer threat correlation to significantly reduce the risk of false positives while ensuring adequate resources to deal with spikes in log volumes.


Don’t get me wrong. Cloud is not a panacea for all big data issues. There are many factors that have to be carefully considered before letting your data rest on the shoulders of the mighty cloud. Data privacy and ownership, data retention costs, cloud provider SLAs are just to name a few. However, there are quite a few of cases where cloud based services can help you manage and make sense of the data and I think it is safe to say that we’ll see more of those services in the future.


When good admins go bad

April 12, 2012

Let’s face it – security breaches will happen. The main question is when. The recent data breach report from Verizon just reinforces this statement with unprecedented growth of security attacks reported across the board. Continue here.

Use case: Tracking user logons across the heterogeneous enterprise IT

March 2, 2012

Quest InTrust and Authentication Services comprise a killer combination to help you track user logons in the IT environment where both Windows and Unix servers and desktops are employed. Continue here.

Latest and greatest InTrust version 10.4 comes out of the door

January 25, 2012

The version 10.4 of InTrust advances in all three main capabilities attributed to successful event log management products. Continue here.

CloudCamp experience

November 11, 2010

I wanted to drop a note about an unusual cloud event I recently attended in San Francisco.

CloudCamp is a half day conference related to trends in cloud technologies. It’s held throughout the world and supported by well known experts and companies in the cloud industry.
It was a 4 hour event packed with a lot of interesting content and discussions. I liked the idea that attendees could submit topics for discussions and a self nominated panel of cloud experts could pick up any questions they felt comfortable answering over the beer :).
The event started with vendor sponsored enlightening talks. Speakers were teaching the audience what’s unique about the cloud restraining from pitching their own tools.
Scalr and RighScale shared VM pattern based approaches they use to efficiently scale cloud applications up and out. Jeremiah Peschka from Quest Software raised the importance of NoSQL databases and explained how Toad for Cloud Databases can help developers interact with cloud databases in the way they’ve been doing with traditional RDBMs.
Adrian Cockroft from Netflix talked about the company’s experience with the recent infrastructure migration to Amazon EC2 (which turned out being a complete rewrite) and what they would like to see provided by yet to mature NoSQL databases. Other presentations were delivered by guys from IBM, Tropo and Twilio.

Everyone agreed that security, reliability and NoSQL will continue being the cloud buzz words throughout 2011.

The break out session on PaaS held by Sebastian Stadil was of particular interest to me. Today most of the cloud derived value comes from IaaS while PaaS still has to go a long way to become an attractive alternative to traditional development platforms.
The group was debating when leveraging PaaS can make sense today. The agreement was the PaaS might be a good start for a “two guys in the garage” startup to quickly spin up their startup project. It becomes more difficult to stick with the PaaS of choice when the service grows and you have to keep up with the increasing demand at the same time staying profitable and not locking yourself into a single platform vendor.

This is when open source projects can play very important role to bridge the gap. Announced at the event the project typhoonae is an open source implementation of Google AppEngine API that can run in any virtual environment. Its flexible architecture let’s you plug in any NoSQL database as a storage backend while making transparent the migration from Google AppEngine PaaS to your choice of IaaS. So, if down the road you’ve learned that for your type of application immediate data consistency can be traded off in favor of constant data availability you could easily switch from say Google Big Table to Cassandra or MongoDB running in Amazon EC2.

Apparently, over the course of the next couple of years we’ll continue seeing the convergence of IaaS and PaaS . For cloud providers this is the only viable way to provide a rapid development platform for highly available, massive data processing applications and cut the cost of on premise applications migration. What an interesting age we’re living in!


April 30, 2010

Hi there!

I’m very excited yet a little bit confused to start my professional blog. The main idea of the blog is to explode and develop my areas of interest to which I include Software as a Service  (SaaS),  Security Information Event  Management (SIEM), Information Security and Systems Management in general.

Today I’m a Senior Program Manager at Quest Software, smart systems management company. My job lets me stay on the edge of new technologies and explore what hides behind such buzz words as Cloud Computing became these days. I hope you’ll be hearing a lot from me on this topic later on.

I hope that someone who will happen to read this blog will find the content useful.   After all I don’t find it worthwhile to write for the sake of writing. So, your feedback and comments are greatly appreciated.

Stay tuned.