Handling the flow of growing amounts of messy data from multiple sources throughout the enterprise is a complex process. Whether moving transactional data so that it can be reported upon, migrating application data from old systems to new ones, or integrating data from external suppliers or partners, the reliable management of data streams can present a challenge to IT organizations. Because these movements of data can be scheduled on a regular basis or executed based on specific triggers, ETL and data warehousing processes inherently lend themselves to automation. Automation brings a host of benefits to these processes including faster delivery times, improved productivity, reduced cost, decreased risk of error, and higher levels of data quality, among others. But the automation tools commonly available often don’t provide the flexibility that is needed for granular and end-to-end automation. For most organizations a different approach, a unified workload automation solution, is needed to achieve all the benefits that automation can provide.
We had the opportunity to talk with many PowerCenter users at Informatica World in Las Vegas in May and they shared a lot of information about their IT Automation requirements, at the application and enterprise level, and how it could be improved. Application vendors, as we have previously addressed in our earlier posting, focus their resources (money and time) in developing the core capabilities of their products rather than enhancing their solutions with a robust automation system necessary to address today’s business and operational requirements.
In recent years, the democratization of analytic, reporting and BI solutions has become a driving force in the growing complexity of data integration and data warehousing models. Add to the equation the growing complexity and volume of information thanks to Big Data, and it’s no surprise that the underlying ETL and data warehousing processes to integrate and access data from multiple sources is becoming increasingly complex.
Attending Informatica World presents the opportunity to speak with IT professionals about one of the strongest use cases for job scheduling and workload automation: the end-to-end automation of ETL, data warehousing and business intelligence (BI) processes.