Test Your Disaster Recovery Strategies before Disaster Strikes

I’m sure you have heard—if not experienced—the following scenario. A student is working on a research paper and suddenly her PC crashes. Because she did not follow the golden rule of saving your document every sixty seconds, she lost hours of work. 

You would think by the time you are well into your career things like this would no longer happen, but unfortunately this kind of thing still happens all the time. When it comes to data, the one thing most people think about is backups. As long as the backups complete without error you feel safe, as you believe you have all of the files you need to restore your database in the event of a disaster. 

But what if I told you that is not a complete disaster recovery strategy?  

We saw this issue play out recently when we were contacted by a company that needed our assistance. The client was trying to restore a database after temporarily losing power and encountered a software bug that was requesting an old archive log that was applied to the database, which happened to be a standby database. Because the archive log requested was so old (a search of emailed backup job logs found the archive log was backed up 9 months prior) and their retention policy only saved backups for 14 days, there was no way for them to get the archive log back. This meant they were not able to restore their data. Long story short: the company lost all of the data in the database.

When one side of our disaster recovery strategy is working we often overlook the second side of the strategy, which is making sure we are able to restore our database using the files created. While the backup job may complete without errors, file corruption or erroneously deleting one of the backup files can render your recovery plan and data useless. This is why we here at Buda Consulting always recommend that our clients perform biannual disaster recovery restore tests at a minimum, with quarterly disaster recovery restore tests at a maximum. 

As the old saying goes, “It’s better to be safe than sorry,” and testing your disaster recovery data is essential to keeping you and your data safe!

Concerned (as you should be) about your disaster recovery and business continuity capability? Contact Buda Consulting to schedule a free consultation.

It’s in the Database

Managed Health Systems of Indiana patient health information, July-September, 2019; Microsoft customer service and support records, January 22, 2020; Wyze email addresses, December 30, 2019; Georgia Tech student data, March 2019.

What do all of these breaches have in common? The data that was stolen was inside a database.

Yet when most companies think about data security, they still focus on securing the network, and spend very little time and energy making sure the databases—where the data actually lives—are safe.

When was the last time you had a network security assessment done? If yours is like most companies, it was pretty recently… and that’s good.

But when was the last time you had a database security assessment done? If yours is like many companies, the answer is “Never.”

Even if your network security posture is robust, it is only a matter of time before your network is breached. And don’t forget about “insider threats,” both malicious and accidental. It is best to add another layer of protection between the bad actors and your data.

Make sure that when cybercriminals do get past your network protections, your database will keep them out.

Download our Database Security Roadmap get valuable insights into create an in-depth defensive posture to protect your data.

Communication Breakdown: Avoiding Conflict with Your Customers

When working with customers, we face many situations that might result in conflict. This is bad for obvious reasons. As service providers, conflict keeps us from providing outstanding value to our customers. It hurts our relationship and reduces the chance that they will continue to use our services.

Customers are at different technical levels and have different “personal styles.” Some are very technical and want to be part of designing the solution. Others won’t want to know any details and just want something that works at the end. Until we get to know the customer well, the potential for conflict is higher for reasons like these.

I have certainly not mastered the art of avoiding all conflict with every customer. But I continue to strive to improve. This post describes two kinds of situations where conflict is likely to occur if we don’t handle things well, along with some thoughts about how to avoid conflict in these and other instances.

The customer is always right!

Of course, we know that the customer is not literally always right. We also know (or had better know) that we are not always right, either.

But even though the customer is not always right, we must always respect them as if they are right. Why? Because it is the right way to treat people, and because they pay us. They are the reason we are in business.

We have all had times when a customer wanted us to do something that we felt was not in their best interest, was not efficient, or was just plain wrong.

So how do we handle it when we think our client is wrong? Below are some basic “rules of conduct” for reaching agreement. It’s good practice to follow these in order. Hopefully this will lead to building great relationships with our clients and avoiding conflicts.

    1. Listen completely to what the customers says, so we fully understand what they mean. This means not interrupting them before they have finished their thought. This takes patience. Often, if we let them talk through an issue long enough, they will come to the same conclusion we would have, and we have just avoided a conflict. Likewise, we may come to understand their position better and we will agree with them; and again, we have avoided a conflict.
    2. If after careful listening there is still not agreement, asking questions may help both parties see aspects of the issue that the other does not see. Try to use probing questions, not leading questions. Our frame of mind should be “How can they convince me that they are right?” Not “How can I convince them that I am right?” In other words, assume they are right, and help them prove it. I am not suggesting that you disregard your knowledge and expertise. But it’s important to acknowledge that you don’t know everything your customer knows. Challenge their position rigorously, but with an open mind.
    3. If there is still not agreement after they have expressed their entire thought and you have asked your questions and listened to the answers, try offering alternative approaches that may meet their objectives, but with methods that you think are more effective.
    4. If you and your customer still don’t agree after all that, then respectfully suggest that you table the issue and follow up via email with additional thoughts (in the same spirit of understanding that was described above).

If, after all your effort, you still think that what the customer is asking you to do is either not in their best interest or is ethically or legally wrong, then you may have to respectfully decline to provide the service they are asking for. I have done this and it’s not the end of the world.

Notice that nowhere did I recommend arguing with the client. Why? Because we never argue with the client. That is disrespectful. Oh, and they pay us.

This approach requires patience—but it’s worth it. Great communications allows us to add more value. Also they pay us, so they deserve it.

This approach might require that we answer the same question ten times. But that is also worth it, because great communications allows us to add more value, and because the client pays us.

This approach might also require that we ask the same question ten times or explain the same thing ten times. But it is worth it. Why? Because great communications allows us to add more value, and because they pay us.

The cranky customer

Right or wrong, the customer might get cranky with us from time to time. But it is worth it to endure this calmly and politely, because they pay us.

This does not mean accepting abuse. If a client is truly abusive, fire them. I’ve done that, and it’s not the end of the world. Service providers deserve and demand the same respect that they give to their customers.

But it does mean that we may have to accept expressions of frustration and even anger at times. When this happens, it is almost always due to problems with communication. And it is up to us to fix the communication problems. Why? Because they pay us.

Here are some simple steps to avoiding communication problems that lead to cranky customers:

    1. Over-communication: When giving an explanation, provide a littlemore information than you think is necessary, as long as that information is useful. If a customer tells you that you should provide less detail, then do so in future communications. It is better to be told you provide too much information than too little.
    2. Frequent communication: During the course of a project, particularly when a deliverable is expected soon, provide short, frequent status updates. The frequency depends on the length and urgency of the project. If the deliverable is due in one week and is urgent, then a daily email with a quick status can put a customer’s mind at ease. If it is a six-month project with ongoing deliverables, then a weekly recap may be enough.
    3. Set expectations with a communication rhythm: if you are in the middle of ongoing work with frequent deliverables, let the customer know that you will check Slack or email twice a day; for example, at 12 and 5. That way they won’t be disappointed or anxious if you don’t respond immediately to their 9:15 email asking for a status update. If you are on a longer-term project, let them know what days of the week you will provide status updates.
    4. Communicate problems or concerns immediately: If your project falls behind schedule, or runs into problems that are beyond your control, inform your customer immediately. This can be a difficult conversation to have, but the sooner they know about a blocking issue, the easier and less expensive it is to resolve it. And the moment you have the conversation, you will feel better!
    5. Be responsive: If your client asks you for something, either respond promptly with the answer, or acknowledge their request as soon as you see it and tell them when you will be able to get to it.

Here is the takeaway: the root of most conflict is misunderstanding, and the root of most misunderstanding is poor communication. As service providers, proper communications are our responsibility. It is part of our job.

We can build all the environments, databases and applications in the world. But if we don’t communicate well with our customers then we have not added the value that we should have.

To get maximum value from your Oracle database investments and clear communications from your service provider, contact Buda Consulting.

Oracle Data Encryption Options

Oracle Data Encryption Options

Oracle offers various authentication and audit features to protect data from unauthorized access. But what about data at rest in operating system files, backups or other storage media?

Protect Oracle Data At Rest With TDE

To protect data at rest, Oracle offers Transparent Data Encryption (TDE). With TDE you can encrypt sensitive data so that it is unreadable if the file it is stored in is exfiltrated or breached.

Data you encrypt with TDE is “transparently” decrypted when it is accessed by authorized users and applications. That is, decryption takes place without users even being aware that data is encrypted. Likewise, applications that process sensitive data can offer data encryption via TDE with little or no code changes.

Why use TDE? It helps ensure that your sensitive data is secure, supports compliance with a wide range of regulations like Sarbanes-Oxley (SOX), HIPAA and PCI, and can simplify your overall encryption/decryption policy and operations.

Another benefit of TDE is that it is pretty fine-grained. You can encrypt data at the column level or the tablespace level. Column-level encryption is perfect for confidential data like social security numbers or credit card numbers that are stored in table columns.

When you encrypt a tablespace, all objects created in that tablespace are encrypted automatically. Tablespace level encryption works well for tables that store sensitive data in multiple columns, or for when you want to protect an entire table and not just individual columns. It’s also handy anytime you want to avoid doing a nitty-gritty analysis of each table column to determine which ones require encryption.

To enable decryption and prevent unauthorized decryption, TDE uses a two-tiered, key-based encryption architecture. It stores encryption keys in a keystore, a hardware or software security module separate from the database. You can centrally (and automatically) manage these keystores using Oracle Key Vault.

To encrypt a tablespace, TDE uses an externally stored master key to encrypt the TDE tablespace encryption key, which is used to encrypt/decrypt tablespace data. For column-level encryption, Oracle transparently accesses a TDE master encryption key to encrypt or decrypt the TDE table key, which then encrypts/decrypts column-level data in the table.

Encryption Best Practices

Of course, your encryption strategy should be integrated with your overall information security program. Best-practice security tips related to encryption include:

      • Start by determining how sensitive the data is. Data that requires the strongest protection can be encrypted using the AES256 algorithm. Conversely, you can encrypt less sensitive data in several ways that offer performance benefits.
      • You also need to determine your approach to keystore protection based on data sensitivity. Options range from auto-login software keystores to hardware keystores. A separate keystore for TDE only is ideal if possible.
      • To limit damage from compromised admin credentials or insider threats, consider assigning separate security admins for TDE and for the database(s).
      • Backup your sensitive data using protected backup procedures.
      • Be aware that column-level encrypted data is decrypted during expression evaluation and could potentially be accessed in the associated on-disk swap file.
      • Also be aware that your Oracle data files could contain plaintext fragments (aka “ghost records” that were deleted logically from the table but still exist physically on-disk. These could potentially be accessed similarly to finding data on-disk after it has been deleted at the operating system level.

For more information on TDE, see the Oracle Advanced Security Guide online.

For expert help and guidance with encryption, backup/recovery, high availability and other business continuity and security concerns, contact Buda Consulting for a security risk assessment—the first step to finding and closing the gaps in your database security.

GDPR Right to Be Forgotten and Other Access Rights

GDPR is complex, and this post deals with only a small part of the law. GDPR is comprised of 99 Articles, of which three (Articles 15, 16 and 17) deal primarily with a consumer’s right to “be forgotten.” This includes the right to access the data that your business keeps about them (Article 15), the right to have incorrect information about them fixed (Article 16) and the right to delete information that your business has collected about them (Article 17).

A summary of the requirements of these GDPR Articles appears at the end of the post. However, the purpose of this article is not to explain these requirements, but rather to suggest approaches that your technology team can take to facilitate compliance with them.

Technical approaches to facilitate compliance

To ease compliance with Article 15, 16, and 17 of GDPR, and to support compliance with other GDPR Articles, we recommend the following steps with regard to handling of personal data. This is not intended to be a comprehensive list of everything you need to do to comply with GDPR. Instead, we see these as best practices to make compliance easier to achieve and more likely to be adhered to. Many of these steps are important for general data security as well—so even if you are not subject to GDPR these are good practices to follow.

  1. Keep an up-to-date data dictionary (metadata) that clearly identifies the location and meaning of all data elements that contain personal data (see definition below). From a GDPR perspective, it will be helpful to keep the following information in this dictionary. This will help when creating your privacy policy.
    • Name of the data element (column name)
    • Where the data element is stored (database name, table name)
    • The meaning of the data element
    • What the data element is used for in the system (i.e., the business reason for needing it)
    • How the data element is collected (what screen or input file)
    • The Personal Data Category from the GDPR regulation (see Article 15 below)
  2. This data dictionary must be updated each time the database is modified to add or remove data elements, or to change the meaning or use of an existing data element (which is not a good practice, but that is a topic for another blog)
  3. Create a set of data entry screens that customer service staff can use to easily perform the actions required under these GDPR articles. These screens can call stored procedures (recommended) or other code that does the work of gathering or modifying the information, and keeping a record of it when appropriate. Using a data entry screen rather than the command line allows a customer service representative to perform these actions rather than relying on manual steps by a developer or DBA. Note that all requests of this type should be validated by sending an email back to the recipient and waiting for confirmation before complying. These screens would call the following:

To support Article 15 — Access

Stored procedures to query all data for an individual, in order to comply with requests from an individual for the data you have about them. By providing this data in a machine-readable format, this can also support compliance with GDPR Article 20 (data portability).

To Support Article 16 — Rectification

One or more stored procedures to query a given user and to modify any personal data that an individual might request.

To Support Article 17 — Erasure

A stored procedure that will cleanly remove all personal data for an individual, as well as related data if the removal of personal data renders any remaining data for an individual useless. If you are using a relational database that has the capability, we recommend creating cascading delete constraints or triggers so removal of related data is simple, safe, and automatic. It is important not to log the removal of any personal data if the log would contain any of the data that was removed.

Depending on your business requirements, there is an alternate technical approach that can help you comply with the Article 17. GDPR requires that data be removed upon request, but that only applies after any legal obligation that your company has to keep the data. For example, if you have to keep contracts for seven years for tax purposes, then you do not have to comply with a request for erasure until that period passes. So an alternative approach to complying with Article 17 would be to write maintenance software that automatically removes all personal information from your system after legal obligations no longer require it. This removes some historical reporting capability, so it may not be appropriate for your business.

Details about the referenced GDPR Articles

The following information about the GDPR right to be forgotten and other rights mentioned above is taken directly from the GDPR recitals as published here: https://gdpr.eu/. In some cases I changed the structure to make it a bit easier to understand the rules.

Pertinent Definitions

  • Personal data means any information relating to an identified or identifiable natural person (“data subject”). An identifiable natural person is one who can be identified, directly or indirectly; in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or mor factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person.
  • Processing means any operation or set of operations which is performed on personal data or on sets of personal data, whether or not by automated means, such as collection, recording, organization, structuring, storage, adaptation or alteration, retrieval, consultation, use, disclosure by transmission, dissemination or otherwise making available, alignment or combination, restriction, erasure or destruction.
  • Controller means the natural or legal person, public authority, agency or other body which, alone or jointly with others, determines the purposes and means of the processing of personal data, where the purposes and means of such processing are determined by Union or Member State law, the controller or the specific criteria for its nomination may be provided for by Union or Member State law.
  • Processor means a natural or legal person, public authority, agency or other body which processes personal data on behalf of the controller.
  • Recipient means a natural or legal person, public authority, agency or other body, to which the personal data are disclosed, whether a third party or not. However, public authorities which may receive personal data in the framework of a particular inquiry in accordance with Union or Member State law shall not be regarded as recipients; the processing of those data by those public authorities shall be in compliance with the applicable data protection rules according to the purposes of the processing.

Article 15: Right of Access By The Data Subject

The data subject shall have the right to obtain from the controller confirmation as to whether or not personal data concerning him or her are being processed, and, where that is the case, access to the personal data and the following information:

  • the purposes of the processing;
  • the categories of personal data concerned;
  • the recipients or categories of recipient to whom the personal data have been or will be disclosed, in particular recipients in third countries or international organizations;
  • where possible, the envisaged period for which the personal data will be stored, or, if not possible, the criteria used to determine that period;
  • the existence of the right to request from the controller rectification or erasure of personal data or restriction of processing of personal data concerning the data subject or to object to such processing;
  • the right to lodge a complaint with a supervisory authority;
  • where the personal data are not collected from the data subject, any available information as to their source;
  • the existence of automated decision-making, including profiling, referred to in Article 22(1) and (4) and, at least in those cases, meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject.
  • Where personal data are transferred to a third country or to an international organization, the data subject shall have the right to be informed of the appropriate safeguards pursuant to Article 46relating to the transfer.
  • The controller shall provide a copy of the personal data undergoing processing. For any further copies requested by the data subject, the controller may charge a reasonable fee based on administrative costs.
  • Where the data subject makes the request by electronic means, and unless otherwise requested by the data subject, the information shall be provided in a commonly used electronic form.
  • The right to obtain a copy referred to in paragraph 3 shall not adversely affect the rights and freedoms of others.

Article 16: Right to Rectification

The data subject shall have the right to obtain from the controller without undue delay the rectification of inaccurate personal data concerning him or her. Taking into account the purposes of the processing, the data subject shall have the right to have incomplete personal data completed, including by means of providing a supplementary statement.

Article 17: Right to Erasure (right to be forgotten)

When does the right to be forgotten apply?

  • where the personal data are no longer necessary in relation to the purposes for which they are collected or otherwise processed
  • where a data subject has withdrawn his or her consent or objects to the processing of personal data concerning him or her
  • where the processing of his or her personal data does not otherwise comply with this Regulation
  • That right is relevant in particular where the data subject has given his or her consent as a child and is not fully aware of the risks involved by the processing, and later wants to remove such personal data, especially on the internet. The data subject should be able to exercise that right notwithstanding the fact that he or she is no longer a child.

Exceptions:

  • The further retention of the personal data should be lawful where it is necessary, for exercising the right of freedom of expression and information
  • for compliance with a legal obligation
  • for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller
  • on the grounds of public interest in the area of public health
  • for archiving purposes in the public interest, scientific or historical research purposes or statistical purposes
  • or for the establishment, exercise or defense of legal claims.

And further:

  • the right to erasure should also be extended in such a way that a controller who has made the personal data public should be obliged to inform the controllers which are processing such personal data to erase any links to, or copies or replications of those personal data. In doing so, that controller should take reasonable steps, taking into account available technology and the means available to the controller, including technical measures, to inform the controllers which are processing the personal data of the data subject’s request.

Disclaimers

This post is based on my reading of GDPR and technical approaches that I feel will facilitate compliance as I understand it.

This is not intended to be a comprehensive discussion or instructions for complying with GDPR.

After taking all technical steps you feel are necessary to comply, you should consult an attorney to determine if the steps you have taken are sufficient for compliance.

For expert help bringing your database environment into compliance with GDPR, CCPA or other emerging privacy legislation, contact Buda Consulting.

 

 

Database Patch News — November 2019 (Issue 1)

Database Patch News — November 2019 (Issue 1)

Welcome to Database Patch News, Buda Consulting’s monthly newsletter of current patch information for Oracle and Microsoft SQL Server. Here you’ll find information on available patches—including security patches—and desupported versions made available during the past month.

Why should you care about patching vulnerabilities and bugs? Two big reasons:

  1. Unpatched systems are a top cyber attack target. Patch releases literally advertise vulnerabilities to the hacker community. The longer you wait to patch, the greater your security risk.
  2. Along with running a supported database version, applying the latest patches ensures that you can get support from the vendor in case of an issue. Patching also helps eliminate downtime and lost productivity associated with bugs.

Here are the latest patch updates for Oracle and SQL Server:

Oracle Patches:

Oct 15 2019 Quarterly Patch Updates:

19c – Release Update 19.5 available

18c – Release Update 18.8 available

12.2.0.1 – OCT 2019 RELEASE UPDATE 12.2.0.1.191015 available.
Regular support ends Mar 2023 and extended support ends Mar 2026.

12.1.0.2 – Currently in extended support.
The last freely available patch was July 2019 for 12.1.0.2. The Oct 15 2019 Patch Set Update (PSU) is available but may require extended support purchase to access it. Patches will be release until July 2021 for this version. PSU 12.1.0.2.191015 is available.

11.2.0.4 – Entered extended support in December 2017
The last free available patch was October 2018 for 11.2.0.4. PSU 11.2.0.4.191015 is available but may require clients purchase extended support to access it.

SQL Server Patches:
SQL Server 2017 incremental servicing model (ISM)
CU17 (Latest build)—Released October 08, 2019

SQL Server 2016 Service Pack 2
Release date: April 24, 2018

SQL Server 2014 Service Pack 3 Cumulative update 4
Release date: July 29, 2019

SQL Server 2014 Service Pack 2 Cumulative update 18
Release date: July 29, 2019