(/Search)
(/)
STORAGE 05/07/2015 7:00 AM
John Miecielica (/author/36436413) Commentary Connect Directly (https://twitter.com/TeamQuest_Corp)
(https://www.linkedin.com/pub/john-miecielica/b/4a/ba9)
(mailto:
[email protected])
Rating:
0 votes
+ Like 0 Tweet Share
Big Data For IT Operations: Data Lakes Or Data Warehouse? IT operations itself can benefit from the promise of big data analytics, but choosing the right data storage ecosystem is essential. We are solidly in the midst of the era of big data (http://www.networkcomputing.com/data-centers/big-data-deployment-trends/a/did/1319138), along with the big hype that goes with it. The next generation of analytics platforms has been emerging for the last few years, but as we enter the implementation phase, the real work begins. There are a lot of vendors and new tools in the market now, but most experts are predicting consolidation. Likewise, companies that want to use analytics programs to stay competitive are pushing to get beyond the beta phase and start reaping rewards from established, well-planned initiatives. Among the most practical use cases for big data analytics is the innovative mining of IT operational data. IT departments have long been tasked with collecting data in service of automating and optimizing all kinds of business processes. I see this turning inward as more IT teams are collecting petabytes of raw machine, sensor, and log data in hopes of visualizing and optimizing their own operations. Doing anything meaningful with such massive amounts of data is challenging. An ecosystem of IT operations management solutions is bubbling up around the use of open source, Hadoop-based data lakes. As the technology matures, enterprises are moving from storage and batch analytics to streaming, real-time data processing built on flexible, modular platforms. Because early adopters have won visibly significant competitive advantage through their data initiatives, analytics are going mainstream, spurring a new wave of innovative solutions. While the Hadoop ecosystem (http://www.networkcomputing.com/applications/big-datadeployment-models-open-vs-proprietary/a/d-id/1319627) is not suited to every type of data project, it points the way to the intelligent liberation of data. To make big data self-service, it has to be accessible to business end-users, not just data scientists and datacenter gurus. To unlock the true potential of big data, the sharing of large data sets has to become more lightweight and transparent. Solutions that address high cost barriers to entry, vendor lock-in, and ultra-rigid data should have a democratizing effect. As the use of data lakes increases, so do the concomitant challenges. Many are asking why and when it makes sense to deploy a Hadoop-based system instead of an enterprise data warehouse (EDW) model. Gartner has cautioned that data lakes can easily become data swamps, full of dirty data and stagnant from lack of use. As always, security, privacy, and compliance concerns are front and center; making a Hadoop environment ready for sensitive information requires custom hardening and configuration. Hadoop-based deployments still require hardware and software installation and management; and new skill sets are needed to integrate modules and applications like Hadoop Common, HDFS, YARN, MapReduce, NoSQL, and analytic discovery. UPCOMING INDUSTRY EVENT Cloud Track at Interop ITX
(http://adclick.g. doubleclick.net/p cs/click%253Fxai %253DAKAOjsuY 1VyIwDylDIZ24zjIl Dk3VqXEq2arhSF t_6FrMlcJyjSa7V HNeqYVRV_aCEi bJ2JONAQgMQJeO Nsd5ELJfB0urYTj 9wx4BjbsC1kug mQDvl3GMtX7XGDAIhw2 xpe3yx2MO4mFIz MDkM1RceCvTvDv1l7j7fvd7 KVsu4nJOaFB_3 dUHnLYHdooYw TFiZjNTuITJkNcq jdeqLOOXtOz5I7l Y_OUwv5gfo1vqz hnsRXcKmAQhY 6khcw%2526sig %253DCg0ArKJS zJgf6DuUqE1EAE%2526url fix%253D1%2526 adurl%253Dhttp:/ /schedule.interop .com/track/infrast ructure/%3F_mc %3Dhsad_x_nwc _le_tsnr_intplv_x _x-ncnativead) (http://adclick.g.doubleclick.net/pcs/click%253Fxai%253DAKAOjsuY1VyIwDylDIZ24zjIlDk3VqXEq2arhSFt_6FrMlcJyjSa7VHNeqYVRV_aC EibJ-2JONAQgMQJeONsd5ELJfB0urYTj9wx4BjbsC1kugmQDvl3-GMtX7XGDAIhw2xpe3yx2MO4mFIzMDkM1RceCvTvDv1l7j7fvd7KVsu4nJOaFB_3dUHnLYHdooYwTFiZjNTuITJkNcqjdeqLOOXtOz5I7lY_OUwv5gfo1vqzhnsRXcKmAQhY6khcw%2526si g%253DCg0ArKJSzJgf6DuUqE1EAE%2526urlfix%253D1%2526adurl%253Dhttp://schedule.interop.com/track/infrastructure/%3F_mc%3Dhsad_x_nwc_le_tsnr_intplv _x_x-ncnativead) Here are the top sessions:
Getting Started with Serverless Breaking Out of the Cloud Providers' Walled Gardens See the Entire Cloud Agenda
Data lakes are an easier and faster way to park and process massive amounts of unstructured data from multiple sources; the most salient feature of Hadoop is that it doesn’t require schema-on-write. This is a timely solution for companies that know they have a lot of valuable data but aren’t quite sure what to do with it yet. Data scientists will also benefit greatly from running experiments in such an open and evolving framework. But depending on the data type, use case, or desired outcome, the lack of structure can be a major drawback. The information being added to a data lake carries no metadata, and without a modicum of curating and governance, it is hard to determine the quality and provenance of the data. Data warehouses, on the other hand, sanitize and organize data upon entry, enabling consistent and predictable analysis across pre-categorized structures. The ability to replicate standard queries and reports over time across uniform datasets is essential to many enterprises. In other words, data warehouses provide value that will not be replaced by data lakes, no matter how flexible they are. With either approach, and regardless of which platform or tools you ultimately deploy, getting the basics right is essential. Storing and accessing data elegantly doesn’t necessarily solve business problems or boost the bottom line. Measuring and analyzing the right things, asking the right questions, and involving the right stakeholders are always keys to success. How do we know if we are measuring the right things? This is where IT and business leaders must cooperate and keep the focus on business needs. Once a meaty business problem has been identified and assessed, it is easier to pick which tools are better suited to building a solution. Sometimes it will involve analyzing data in rigid silos; sometimes it will be drawing samples from a fluid pool of data. As next-generation analytics continue to evolve, we will no doubt invent new approaches that blend these models to achieve even deeper levels of knowledge, to the benefit of the data center and beyond.
We welcome your comments on this topic on our social media channels, or [contact us directly] (https://www.networkcomputing.com/about-us) with questions about the site. EMAIL THIS (/printmail/1320313)
PRINT (/print/1320313)
RSS (/rss/all)
MORE INSIGHTS Webcasts IT Security Strategy: What to Keep in House vs. What to Outsource (https://webinar.darkreading.com/3587? keycode=sbx&cid=smartbox_techweb_webcast_8.500000824) Cybersecurity Crash Course - Session 7: Security For IoT (https://crashcourse.darkreading.com/2882? keycode=sbx&cid=smartbox_techweb_webcast_8.500000796) MORE WEBCASTS (/webinar_upcoming)
White Papers GDPR Without the Hype (https://www.darkreading.com/endpoint/privacy/gdpr-without-the-hype/d/d-id/1331471? cid=smartbox_techweb_whitepaper_14.500003199) ISA Delivers Major, Ongoing ROI (http://www.informationweek.com/whitepaper/risk-management-security/risk-management/information-securityawareness-delivers-major,-ongoing-roi/396033?cid=smartbox_techweb_whitepaper_14.500003193) MORE WHITE PAPERS (http://www.informationweek.com/whitepaper/Infrastructure)
Reports [Dark Reading Report] Navigating the Threat Intelligence Maze (http://www.informationweek.com/whitepaper/cybersecurity/risk-managementsecurity/[strategic-security-report]-navigating-the-threat-intelligence-maze/393933?cid=smartbox_techweb_report_7.300005741) 2017 State of IT Report (http://reg.interop.com/stateofit?kcode=nwc_rptbx&cid=smartbox_techweb_report_7.300005737) MORE REPORTS (http://www.informationweek.com/whitepaper/search?querytext=&search-results-topics=infrastructure&search-results-subtopics=&searchresults-company=53472&startdatetimepicker=&enddatetimepicker=&search-results-format-researchreport=on)
SUBSCRIBE TO NEWSLETTERS (/user)
SLIDESHOWS
6 Ways to Recycle Your IT Gear for Earth Day (/data-centers/6-ways-recycle-your-it-gear-earth-day/1719339574) Read (/data-centers/6-ways-recycle-your-it-gear-earth-day/1719339574) Post a Comment (/data-centers/6-ways-recycle-your-it-gear-earth-day/1719339574#comment-form) IaaS Cloud Adoption Trends (/data-centers/iaas-cloud-adoption-trends/1128344352) 5 Steps for Government Cybersecurity (/network-security/5-steps-government-cybersecurity/1779295039) MORE SLIDESHOWS (/slideshows)
CARTOON
(/network-security/maximum-network-
security/1895193444) CARTOON ARCHIVE (/cartoons)
WEBINARS IT Security Strategy: What to Keep in House vs. What to Outsource (https://webinar.darkreading.com/3587? keycode=sbx&cid=smartbox_techweb_webcast_8.500000824) Bulletproof Your Digital Footprint From Emerging Threats (https://webinar.darkreading.com/3534? keycode=sbx&cid=smartbox_techweb_webcast_8.500000827) Cybersecurity Crash Course - Session 7: Security For IoT (https://crashcourse.darkreading.com/2882? keycode=sbx&cid=smartbox_techweb_webcast_8.500000796) WEBINARS ARCHIVES (/webinar_archives)
LIVE EVENTS
WHITE PAPERS GDPR Without the Hype (https://www.darkreading.com/endpoint/privacy/gdpr-without-the-hype/d/d-id/1331471? cid=smartbox_techweb_whitepaper_14.500003199) ISA Delivers Major, Ongoing ROI (http://www.informationweek.com/whitepaper/risk-management-security/risk-management/information-securityawareness-delivers-major,-ongoing-roi/396033?cid=smartbox_techweb_whitepaper_14.500003193) 451 Research: The Emergence of Unified Access Management (http://www.informationweek.com/whitepaper/cloud-services/software-as-aservice/451-research-the-emergence-of-unified-access-management/396583?cid=smartbox_techweb_whitepaper_14.500003217) 4 Reasons DLPs Aren't Cutting It (http://www.informationweek.com/whitepaper/database-security/security-management-and-analytics/4-reasonsdlps-aren%E2%80%99t-cutting-it-/396413?cid=smartbox_techweb_whitepaper_14.500003211) Spies Among Us: The Rise of State Sponsored Insider Threats (http://www.informationweek.com/whitepaper/database-security/securitymonitoring/spies-among-us-the-rise-of-state-sponsored-insider-threats/396403?cid=smartbox_techweb_whitepaper_14.500003210) MORE WHITE PAPERS (http://www.informationweek.com/whitepaper/Infrastructure)
CURRENT ISSUE
(http://www.networkcomputing.com/nwcdigital/20171107?
cid=smartbox_techweb_nwcdigital_20171107)
2018 State of Infrastructure Report (http://www.networkcomputing.com/nwcdigital/20171107? cid=smartbox_techweb_nwcdigital_20171107) DOWNLOAD THIS ISSUE! (http://www.networkcomputing.com/nwcdigital/20171107?cid=smartbox_techweb_nwcdigital_20171107)
BACK ISSUES (/backissue-archives)
MUST READS (/mustreads)
VIDEO (/datacenters/containersvs-paas-toughchoice/672134063? itc=AD_NWC_VID_ RHC_VIDBOX) Containers Vs. PaaS: A Tough Choice (/datacenters/containersvs-paas-toughchoice/672134063? itc=AD_NWC_VID_ RHC_VIDBOX) ALL VIDEOS (/videos)
(/data-centers/liftand-shift-viablecloud-migrationstrategy/1595012945 ? itc=AD_NWC_VID_ RHC_VIDBOX) Is Lift and Shift a Viable Cloud Migration Strategy? (/datacenters/lift-andshift-viable-cloudmigrationstrategy/159501294 5? itc=AD_NWC_VID_ RHC_VIDBOX)
(/networking/powernetworkdisaggregation/1878 422733? itc=AD_NWC_VID_ RHC_VIDBOX) The Power of Network Disaggregation (/networking/power -networkdisaggregation/187 8422733? itc=AD_NWC_VID_ RHC_VIDBOX)
(/storage/doeshyperconvergedinfrastructure-savemoney/1709320553? itc=AD_NWC_VID_ RHC_VIDBOX) Does Hyperconverged Infrastructure Save Money? (/storage/doeshyperconvergedinfrastructure-savemoney/1709320553 ? itc=AD_NWC_VID_ RHC_VIDBOX)
(/networking/tomhollingsworthnetworkingstransitionsoftware/377321358 ? itc=AD_NWC_VID_ RHC_VIDBOX) Tom Hollingsworth on Networking's Transition to Software (/networking/tomhollingsworthnetworkingstransitionsoftware/377321358 ? itc=AD_NWC_VID_ RHC_VIDBOX)
REPORTS [Forrester's Report] The State of Application Security: 2018 & Beyond (http://www.informationweek.com/whitepaper/riskmanagement/security-management-and-analytics/forrester's-report-the-state-of-application-security-2018-andbeyond/394673?cid=smartbox_techweb_analytics_7.300005742) DOWNLOAD NOW! (http://www.informationweek.com/whitepaper/risk-management/security-management-and-analytics/forrester's-report-the-state-of-application-security-2018-and-beyond/394673? cid=smartbox_techweb_analytics_7.300005742)
MORE REPORTS (http://www.informationweek.com/whitepaper/search?querytext=&search-results-topics=infrastructure&search-results-subtopics=&searchresults-company=53472&startdatetimepicker=&enddatetimepicker=&search-results-format-researchreport=on)
TWITTER FEED NetworkComputing Retweeted
Peter Jones @petergjones
The network must enable the business to succeed. #InteropITX By @dconde_sf in @NetworkComputin Mike Kazemian Retweeted
Savvius Inc @SavviusInc
“Following the @interop and @InformationWeek State of the Cloud Survey, @marciasavage of @NetworkComputing outlines findings on how organizations are moving and managing workloads to the cloud. See what they found: hubs.ly/H0bKx3V0 by @networkcomputin
ABOUT US (/about-us) CONTACT US (/contact-us) REPRINTS (http://www.wrightsreprints.com/reprints/?magid=2200)
15h
TWITTER (https://twitter.com/networkcomputin) FACEBOOK (https://www.facebook.com/networkcomputingcom) LINKEDIN (https://www.linkedin.com/groups/4403419) GOOGLE+ (https://plus.google.com/+Networkcomputingcom/posts) RSS (/feeds)
(http://www.ubmtechweb.com/)
TECHNOLOGY GROUP Black Hat (http://www.blackhat.com/us-14/)
Enterprise Connect (http://www.enterpriseconnect.com/)
ICMI (http://www.icmi.com/)
Content Marketing Institute (http://contentmarketinginstitute.com/)
GDC (http://www.gdconf.com/)
InformationWeek (http://www.informationweek.com/)
Content Marketing World (http://www.contentmarketingworld.com/)
Gamasutra (http://www.gamasutra.com/)
INsecurity (http://insecurity.com/)
Dark Reading (http://www.darkreading.com/)
HDI (http://www.thinkhdi.com/)
Interop ITX (http://www.interop.com)
Network Computing (http://www.networkcomputing.com/) No Jitter (http://www.nojitter.com/) Service Management World (http://www.smworld.com/) XRDC (http://www.xrdconf.com/) COMMUNITIES SERVED
WORKING WITH US
Content Marketing (http://tech.ubm.com/community-brands/content-marketing-2/)
Advertising Contacts (http://createyournextcustomer.techweb.com/contact-us/)
Enterprise IT (http://tech.ubm.com/community-brands/enterprise-it/)
Event Calendar (http://events.ubm.com/?company=10)
Enterprise Communications (http://tech.ubm.com/community-brands/enterprise-communications/)
Tech Marketing (http://createyournextcustomer.techweb.com/)
Game Developers (http://tech.ubm.com/community-brands/game-and-app-developers/)
Solutions (http://createyournextcustomer.techweb.com/)
Information Security (http://tech.ubm.com/community-brands/information-security/)
Contact Us (http://tech.ubm.com/contact-us/)
IT Services & Support (http://tech.ubm.com/community-brands/technical-service-and-support/)
Licensing (https://wrightsmedia.com/sites/ubm/index.cfm)
Terms of Service | Privacy Statement | Legal Entities | Copyright © 2018 UBM, All rights reserved