by Scott Loudon | Sep 15, 2020 | Cloud, database monitoring, virtual storage
A DBA’s Transition Guide for Hosting on the AWS Cloud
So your organization has decided to migrate your traditional on-premises IT infrastructure to the AWS Cloud in the hopes of realizing cost savings, and to cut down on the time it takes to provision and configure services to support new and changing application workloads. Applications can evolve over time to cloud-centric architectures in order to realize cost savings. But what about all the extra administrative tasks and pressures that go along with the additional speed and agility that cloud hosting provides? How do you keep a handle on all the new instances and know when there are server sprawl issues? Or, even better, avoid server sprawl issues in the first place?
Every DBA knows that whenever anything goes wrong it is always the database that is guilty until proven innocent. So how can DBAs adapt to the new challenges of AWS hosting to remain valuable assets to our organizations?
For the purposes of this blog we will focus on database monitoring and management using the AWS CloudWatch service. CloudWatch ingests performance data from a wide range of AWS resources, applications and services, sends alerts when needed, and keeps a 15-day historical record of performance information. You can even configure CloudWatch with alarm actions to automatically take corrective measures in response to certain predefined event types (but that is a blog for another time). As an added bonus, the CloudWatch “free tier” should be sufficient to perform the heavy lifting of issue detection and identification for most application databases.
Monitoring Performance Metrics of Databases Managed with Amazon RDS
As with traditional on-premises databases, CPU utilization and available memory are two sides of the same performance tuning coin for databases in the AWS Cloud.
You can use the CPUUtilization metric in CloudWatch to keep a historical view of CPU usage for databases managed with Amazon Relational Database Service (Amazon RDS). To get a more complete picture of how an RDS database instance is performing, you can combine CPU monitoring with these additional metrics:
- FreeableMemory, which shows the amount of available memory
- SwapUsage, which shows how much data in memory is being paged to disk due to memory shortages
You can also configure CloudWatch to send alerts when thresholds are crossed.
One of the best features of cloud hosting is you are no longer locked into a specific database footprint based on hardware that was purchased. If you start to see a trend of CPU availability consistently running above 80%, or you’re seeing a shortage of free memory, it could be time to take advantage of the cloud’s on-demand scalability and plan to grow your DB instance to increase capacity. Likewise, if you notice that your databases are consistently showing a large amount of free memory and CPU, then think about scaling down the database instance class to save money.
Storage Monitoring and Auto Scaling To Avoid Server Sprawl
In the AWS cloud, there is never a good reason for running out of available storage on a production database, or any database for that matter. For example, you can use the CloudWatch FreeStorageSpace metric to measure the amount of storage space available to a database instance and trigger alerts as needed. Amazon RDS hosted databases also support storage auto scaling on all major RDS database offerings. This option automatically increases the storage by 5 GB or 10% of currently allocated storage, whichever is higher.
The amount of input/output operations per second (IOPS) for a given database is derived from the storage type you are using together with the amount of storage allocated. It is important to know what IOPS numbers your current storage supports, and you can define the CloudWatch metrics ReadIOPS and WriteIOPS to notify you if you are approaching that level to avoid an issue.
You can get additional IOPS by moving to faster storage or growing your storage footprint to a certain degree. If you exhaust those options and are certain that poor application coding is not leading to excessive read/write activity, it may be time to start thinking about moving to the Provisioned IOPS (PIOPS) storage type, which can provide a higher level of guaranteed I/O for an additional cost.
CloudWatch also offers metrics for ReadLatency, WriteLatency, and DiskQueueDepth for you to configure if you want to keep a closer eye on those parameters.
Monitoring Database Connections
The CloudWatch DatabaseConnections parameter lets you monitor the number of active connections to your database and can alert you when the value approaches the max_connections property for the database.
The default value for max_connections is derived from the total memory and is database-specific, so it is important to check the setting for each database. You can also modify the default value of this parameter if required.
As you can see, CloudWatch simplifies a number of key database monitoring and management tasks. But CloudWatch is just one of several DBA support options you can try on AWS Cloud. You can also subscribe to Amazon RDS events to be notified about changes to a database instance, leverage the Performance Insights dashboard to help analyze database issues, and more.
If your company is thinking of migrating your databases to a cloud or managed hosting provider, Buda Consulting can help you choose the best option for your workloads, and can act as your “first line of defense” when problems like server sprawl arise. We also offer “personalized” managed database environments for Oracle, SQL Server and MySQL workloads.
Contact us to schedule a free consultation today.
For more information:
by Robert Buda | Mar 20, 2019 | Best Practices, licensing, Oracle, Oracle DBA
Virtualization technology revolutionized the RDBMS world by allowing database administrators and system administrations to utilize 90% of a server’s hardware resources, versus on average between 10%-30% utilization in non-virtualized environments. But with the push from a physical to a virtualized environment comes a host of new management tasks. One of those tasks is trying to demystify your licensing.
Oracle Virtual Server Licensing
In most circumstances, your virtualized environment would be running inside a virtual hypervisor. Your virtual servers would then be allocated a set of resources from a pool set in the hypervisor manager. One of the most important settings is how many virtual CPUs (vCPUs) your server is allocated.
The reason vCPUs are such an important virtual resource is because this setting directly relates to the physical processor and its cores. Processor cores is the metric many vendors like Oracle and Microsoft use to license some of their software stacks. This post focuses on Oracle virtual server licensing, which can be licensed by processor core, and explains one of the most common mistakes administrators make when setting up an Oracle environment in a virtualized environment.
A key point to understand is how the database software is licensed. Most servers purchased these days are running Intel Xeon processors containing huge core counts of 12 or higher. Let’s say you purchased a server with two Intel Xeon 4-Core processors. Oracle grades most Intel Xeon processors as 0.5 of a core license each. That means that 2 Xeon cores will equal 1 core license. So, for a system with two Xeon 4-core processors, which is 8 processors total, you will need to purchase an Oracle Database core license count of 4.
But what happens when you decide to upgrade to new servers with two Xeon 12-core processors for less than the original cost of the two Xeon 4-core processors? Oracle requires that all processor cores contained in a server are licensed for their database product when using core licensing. If you were running your database environment in a physical format, you would need to purchase 8 more core licenses, causing your operating cost to increase.
The only way to continue with your 4 core licenses would be to virtualize your database environment—and the only way to comply with Oracle in an on-premise setting outside of an Oracle engineered system is to run Oracle VM.
Oracle VM is Oracle’s own virtualization hypervisor, and it is the only hypervisor where they honor the CPU pinning option, also called processor affinity. This option allows you to permanently allocate a physical processor’s core to a vCPU on a virtual server. Once a core is “pinned” to a vCPU on a virtual server, no other virtual server will use that processing core.
This last point is very important, because most administrators who are familiar with creating virtual servers know that you will be asked the following question to determine if you are compliant with your Oracle virtual server licensing:
- How many vCPUs would you like to allocate to the virtual server?
vCPU Allocation
In most environments, 1 vCPU is equal to 0.5 of a physical core. Do you notice a trend yet? If we wanted to create a virtual server that complied with an Oracle Database licensing core count of 4, the virtual server would need 8 vCPUs.
But what happens if we create the virtual machine with the correct amount of cores and never enable the CPU pinning option? According to Oracle documentation, this means you are not in compliance with your core licensing; and, in order to be in compliance, you will need to guarantee that this virtual machine only has access to the amount of cores specified in your core licensing.
To correct this problem, you will need to go into Oracle VM’s VM Tool software and assign physical CPU cores 0 to 7 (core numbering starts at 0) in order to lock the virtual server to only be able to use 8 cores. Keep in mind that this virtual server will be the only server allowed to use cores 0 to 7, and any other virtual servers hosted by the hypervisor will use physical cores 8 to 23 (remember you upgraded your servers to two 12-core Xeon processors).
After pinning the licensed amount of cores to the virtual server hosting your database environment, you are now in compliance with Oracle licensing and will avoid any costly penalties resulting from an Oracle software audit where you are found to be out of compliance.
So when building out your virtual environments, make sure you know the requirements of your software’s licensing. Forgetting to pin CPUs to your virtual servers could end up costing your company significant money.
If you like this article, please share it with your colleagues and subscribe to our blog to get the latest updates.
by Robert Buda | Nov 24, 2010 | Database Architecture, Oracle
In a recent change to its long-standing policy, Oracle Corp. will now support its customers running Oracle Real Application Cluster (RAC) software on VMware platforms in certain circumstances.
This positive move for customers was announced in a document titled “Support Position for Oracle Products Running on VMware Virtualized Environments,” which Oracle posted on its support web page on November 8. You can read the specifics of the new policy there.
Basically, Oracle will now accept service requests (Srs) for Oracle RAC 11.2.0.2 and later releases running on VMware-based virtual servers, provided the customer can demonstrate that VMware is not the source of the problem. Otherwise, “…the customer will be referred to VMware for support.”
Oracle hasn’t gone so far as to certify any of its products on VMware, but it will provide support for issues that “… are known to occur on the native OS, or can be demonstrated not to be as a result of running on VMware.”
Oracle RAC is a single database instance deployed across a cluster of Intel X86 servers. For better scalability and fault tolerance, RAC offers load balancing and high availability… at a significant extra cost versus Oracle’s Standard Edition (SE) database. RAC is included with Oracle SE but the cluster hardware is limited to two CPU cores. To run your database across more than two cores, you need to pay for RAC on top of the more expensive Oracle Enterprise Edition (EE) software.
What this means
What prompted this move on Oracle’s part? Ongoing customer pressure. Businesses of all sizes want the advantages of virtualized Oracle databases, like reduced hardware costs, reduced power, rack space and cooling costs, and faster database server provisioning.
Oracle had previously asserted that there were “technical restrictions” when running Oracle RAC on VMWare, and explicitly excluded support. This hindered many organizations – especially midsized businesses, which may not have database expertise in-house – from virtualizing their Oracle databases.
But the fact is that Oracle DBMS runs well in a virtual machine. Hundreds of organizations run Oracle on VMWare every day, and it works fine.
If your business has held back on database virtualization because “Oracle doesn’t support VMware,” you might want to reconsider that decision in light of this announcement.
I invite you to comment on how the availability of support from Oracle might change how you deploy your Oracle databases. Did the explicit lack of support hold you back before, and how quickly will you move to increase your virtual footprint now that limited support is available from Oracle?
