by Robert Buda | Apr 27, 2021 | NoSQL
Since I first blogged about NoSQL back in 2013, the use of “Not Only SQL” databases for mission-critical applications has proliferated across businesses of all sizes and sectors. With trends like big data analytics, cloud, mobile and IoT driving the modern digital enterprise, requirements for greater application performance and scalability combined with falling storage costs increasingly bias the playing field toward non-relational databases.
Is relational database (RDBMS) technology still the “reigning data champion”? Yes, but the gap has closed significantly. NoSQL’s more flexible data storage formats are helping to solve a range of problems that traditional relational databases struggle to address.
What are the top NoSQL use cases these days? Here are five of the most prevalent:
1. Real-time/Near Real-time Big Data Processing
The faster a company can process and act on fresh data, the greater its operational efficiency and business agility, and the greater the bottom-line value of its data. A typical approach to real-time big data processing uses stream processing to ingest new data combined with Apache Hadoop for analyzing historical data, plus a NoSQL database that integrates with both.
Payments processing leader PayPal is a prime example of a leading digital enterprise that processes big data on-the-fly and leverages it in multiple ways. PayPal captures vast quantities of raw clickstream data—more than 20 TB per day!—in multiple formats by processing it through Hadoop and Apache HBase NoSQL databases, and storing it all in the cloud for worldwide access by business analysts and data scientists. Fraud detection, data mining, customer segmentation and delivering personalized ads to customers are just some of the differentiating capabilities that PayPal has built on NoSQL.
2. Internet of Things
As of 2021, it’s estimated that about 46 billion IoT devices, from smartphones to home appliances to healthcare systems to factory sensors to smart vehicles, are now connected to the Internet. The amount of semi-structured data these devices continuously generate adds up to something like 847 zettabytes.
NoSQL databases are better suited than their relational counterparts to scale out to ingest this endless fire hose of diverse data. Freshub, a smart kitchen web platform for food shopping, is one example among many of an application that successfully processes data from huge numbers of IP-connected appliances. The Freshub solution maintains a MongoDB NoSQL cloud database of over 1 million grocery products gleaned from online catalogs in real-time. In this use case, NoSQL is well suited to integrate diverse and unpredictable data schemas from all these sources. NoSQL also scales out horizontally across an arbitrary number of cloud database nodes as Freshub grows its customer and data footprint.
3. Content Management
Online shopping now surpasses brick-and-mortar sales, and “content is king” across thousands of online marketplaces and web storefronts. Online sales leaders curate a selection of multimedia content (including user-generated and social media content like reviews, photos and videos) and deliver it to shoppers “at the moment of interaction.”
NoSQL document databases offer a flexible, open-ended data model that is ideal for storing a mix of structured, semi-structured and/or unstructured content. NoSQL also makes it possible to aggregate data that serves multiple business applications within a single catalog database. Whereas RDBMS with its fixed data models tend to result in the proliferation of multiple, overlapping catalogs for different purposes.
Forbes.com, which lives on viewership and ad revenue, exemplifies the use of NoSQL technology for content management. Forbes quickly built a custom content management system based on MongoDB in just a few months, giving them greater agility—including the ability to incorporate contributor content and analyze social sharing within clickstream data—at a lower cost. The same data store also feeds their mobile site, which now gets 50% of their total traffic.
4. Mobile Apps With Huge Numbers Of Users
Mobile phone and tablet use recently surpassed desktops as the top online platform for searching, shopping and otherwise viewing web content. Interestingly, as much of 90% of mobile data is served via apps and only 10% through browsers, an overwhelming shift in recent years.
Rapidly scaling mobile apps globally to serve tens of millions of users with acceptable performance (think mobile gaming or popular social media apps) often calls for distributed databases, which in turn calls for NoSQL. Flexible NoSQL data models also support rapid app update cycles better than relational data models in many cases.
For these reasons, more and more businesses looking to monetize web content are using NoSQL data stores for their apps. A popular case in point is The Weather Channel, whose MongoDB database instance handles millions of requests per minute while also processing user data and juggling weather updates from tens of thousands of worldwide locations.
5. Enriching The Digital Customer Experience
An engaging differentiating digital customer experience is built on data-intensive, time-critical capabilities like personalization, user profile management and a unified view of the customer across all your touch points. A lot of this demographic, behavioral and logistical data comes from the online clickstream, creating a write-intensive, multi-schema workload that taxes “scale-up” RDBMS infrastructure. A distributed NoSQL database can scale more cost-effectively, manage an ever-growing number of attributes with less administrative hassle, and often delivers lower latency—the Holy Grail of online interactions where you’re trying to get ads, recommendations, coupons, etc. in front of users in real-time.
Multimedia service provider Comcast uses a Couchbase NoSQL platform to deliver a positive customer support experience across multiple lines of business. A core goal of the platform is to capture data from huge numbers of omnichannel interactions (phone calls, online help, chatbots, etc.) and relate it all back to individual customers’ accounts and service status. Scalability and resilience are also critical concerns, as in any customer experience scenario. Especially because the better your solution works, the more customers will use it.
Other NoSQL Use Cases & Conclusion
- Real-time updates and queries
- Discussion thread hierarchy
- Data caching and archiving
- Simple data collection and analysis functions associated with voting and surveys
- Cross over data analysis that cannot be conducted in relational environments
- Online gaming where numerous simple queries need to run in fractions of a second
As these current NoSQL use cases illustrate, the strengths that I highlighted back in 2013, like flexible data models, low latency, ease of delivery/maintenance and the ability to integrate structured, semi-structured and unstructured elements, continue to make NoSQL a preferred choice for “digital transformation” across industries. The ongoing success of NoSQL innovators and early adopters like Netflix, Amazon, Twitter, Facebook and AOL continues to pave the way for new solutions.
Wondering if NoSQL technology is right for your application, how to architect a new NoSQL solution or how to move your current RDBMS to a NoSQL alternative? Contact Buda Consulting for a 15-minute “database discussion” to explore whether we can help.
by Robert Buda | Apr 1, 2015 | Oracle DBA
For any big data effort to succeed, an organization needs to figure out how to combine the right data from the right sources to generate the right insights to achieve its goals. Transactional applications might hold data on customer purchases, for instance. But their browsing patterns, loyalty interactions and responses to tweeted offers are probably in web-based systems. You need to pull all the pieces together to solve the puzzle and exploit new opportunities.
Oracle understands this problem well, as evidenced by the four new products and three new services they’ve announced in recent weeks.
The new products include:
- Oracle Big Data Discovery—variously dubbed “the visual face of Hadoop” and “the foundation for data innovation,” it offers a straightforward, unified way for business users to explore data from multiple sources and then analyze it and share the actionable results… in minutes, says Oracle. The benefits include radically accelerated time-to-value for big data “projects” plus increased participation by a wider range of business users, adding up to bigger insights all around.
- Oracle GoldenGate for Big Data—a Hadoop-based tool that supports streaming of real-time, unstructured data from multiple transaction systems straight into popular big data systems like Apache Hadoop, Hive, HBase and Flume. Essentially it replicates data between systems in real-time in your choice of forms, without impacting source system performance.
- Oracle Big Data SQL 1.1—said to offer a query performance boost of up to 40% over previous versions.
- Oracle NoSQL Database 3.2.5—which includes several new features including new APIs, as well as improved security, usability and performance.
Together these new products “further Oracle’s vision to enable Hadoop, NoSQL, and SQL technologies to work together and be deployed securely in any model—whether public cloud, private cloud or an on-premises infrastructure.” It’s all about “operationalizing insights” by integrating new data sources with existing infrastructure, applications and databases.
The new big data services, all cloud-based are geared toward helping companies leverage big data specifically for marketing:
- Oracle Data as a Service for Marketing is aimed at generating sales leads. It offers a staggering 300 million profiles of business users and companies, which can be used to prospect for new business-to-business customers as well as improve your insight into your customer base and drive smarter cross-channel marketing.
- Oracle Data as a Service for Customer Intelligence is designed to provide a clearer picture of customer feedback on products and services, as well as offer insights into emerging trends or customer concerns. Among other data sources, it uses public information from 700 million social networking messages that Oracle collects daily.
- Oracle Marketing Cloud for Student Engagement offers templates that universities and others can use to attract students and improve retention among enrolled students. It essentially packages for the higher education vertical a range of existing Oracle cloud services. Similar packages are already available for a wide range of verticals including manufacturing, insurance, entertainment, nonprofits and many others.
These new offerings join two recent Oracle acquisitions (I know I promised I’d stop at seven, sorry…):
- The BlueKai platform, “the industry’s leading cloud-based big data platform that enables companies to personalize online, offline and mobile marketing campaigns…”
- Datalogix, whose technology “connects offline purchasing data to digital media to improve audience targeting and measure sales impact.”
All these offerings are aimed at helping Oracle customers advance their big data capabilities faster and with greater ease and success. “More people want to use Oracle software without having to run Oracle software,” said Thomas Kurian, Oracle’s VP of product development, at his Oracle OpenWorld 2014 keynote.
Is your IT department looking to respond to business demands for big data analytics that wring new insights and competitive momentum from your Oracle databases? Do you have the expertise you need in-house to address these new challenges while continuing to maintain current databases and applications? Contact Buda Consulting to discuss options for augmenting your core team with an expert Oracle DBA partner that can help with new demands or backstop everyday processes.
by Robert Buda | Jul 12, 2014 | MySQL, NoSQL, Oracle DBA
Oracle Corp. is currently the world’s second-largest software vendor—and it isn’t going to let a little thing like unstructured data stand in its way. With the recent release of its MySQL Fabric technology, which is meant to meet the demands of cloud- and web-based applications, Oracle is positioning itself to dominate the big data landscape.
Most enterprise data is still stored in relational databases written in SQL. To handle diverse data types and increase the flexibility of database structures, database developers are increasingly employing newer, open source DBMSs, especially MySQL (which Oracle maintains) and more recently NoSQL.
MySQL is currently the world’s most popular open source database. An RDBMS-based SQL implementation designed to support web as well as embedded database applications, MySQL drives some of the world’s largest websites including Google, Facebook, Twitter and YouTube. It has proven to be easy-to-use, reliable and scalable.
Despite the promise it offers for big data and real-time web applications, NoSQL has yet to evolve to deliver enterprise-grade reporting and manageability. MySQL Fabric is designed to solve these problems by delivering the best of NoSQL and SQL/RDBMS.
The new MySQL Fabric open source framework seeks to combine the flexibility of NoSQL with the robust speed of RDBMS. It should also simplify the management and scaling of MySQL databases by making it easy to manage them in groups.
MySQL Fabric offers high availability through failure detection and failover, by automatically promoting a slave database to be the new master if the master database goes down. It also offers enhanced scalability through automated data sharding, a process of separating database tables into multiple sections. Sharding helps you manage MySQL databases that are too large (or frequently accessed) for a single server.
Other key features include:
- Automatic routing of transactions to the current master database, combined with load balancing of queries across slave databases
- Extensions to PHP, Python and Java connectors to route transactions and queries directly to the correct MySQL server, eliminating the latency associated with passing through a proxy
By enabling multiple copies of a MySQL database to work together, MySQL Fabric will make it easier to perform live backups and scale MySQL databases across multiple servers. This, in turn, will make it easier to safely “scale out” MySQL applications in both on-premise and cloud implementations.
The new framework will support the growing use of MySQL for high-traffic, business-critical web applications. MySQL Fabric also positions Oracle strongly against NoSQL databases like MongoDB and MySQL add-on providers like Percona. Prior to the release of MySQL Fabric, DBAs had to write code or buy third-party software to create a MySQL server cluster.
You can download the new framework as part of the MySQL Utilities 1.4.3 package at: http://dev.mysql.com/downloads/fabric/
Note that Oracle also offers the MySQL Cluster version of MySQL, which offers some advantages over MySQL Fabric, such as faster failover times and a two-phase commit to ensure that each transaction is fully recognized.
Contact Buda Consulting to talk over how these technologies can help maximize the performance and reliability of your critical, customer-facing applications.
