Call now: 252-767-6166
Search BC Oracle Sites
Home
E-mail Us
Oracle Articles
Oracle Training
Oracle Tips
Oracle Forum
Class Catalog
Remote DBA
Oracle Tuning
Emergency 911
RAC Support
Apps Support
Analysis
Design
Implementation
Oracle Support
SQL Tuning
Security
Oracle UNIX
Oracle Linux
Monitoring
Remote support
Remote plans
Remote services
Application Server
Applications
Oracle Forms
Oracle Portal
App Upgrades
SQL Server
Oracle Concepts
Software Support
Remote Support
Development
Implementation
Consulting Staff
Consulting Prices
Help Wanted!
Oracle Posters
Oracle Books
Oracle Scripts
Ion
Excel-DB
Don Burleson Blog
Oracle 11g Data Compression Tips for the Database Administrator
Oracle 11g Tips by Burleson Consulting
These is an excerpt from the book "Oracle 11g New Features". Tests show that 11g compression result is slower transaction throughput but creates less writes because of higher row density on the data block. See this benchmark of transparent data encryption.
Also see this quick note on how to compress Oracle tablespaces.
While it is true that data storage prices (disks) have fallen dramatically over the last decade (and continue to fall rapidly), Oracle data compression has far more appealing benefits than simply saving on disk storage cost. Because data itself can be highly compressed, information can be fetched off of the disk devices with less physical IO, which radically improves query performance under certain conditions.
Please note that there is common misconception that 11g table compression decompresses data while reading and holds it in the uncompressed form in the cache. That is not quite correct since one of the salient features of the database compression is that we do not have to uncompress the data before reading it and the data stays in the compressed form even in the cache.
As a result, Oracle table compression not only helps customers save disk space, it also helps to increase cache efficiency since more blocks can now fit in the memory.
According to the Oracle whitepaper on 11g data compression, the CPU overhead for the compress/decompress operations will be minimal. More important, Oracle11g data compression will be a Godsend for shops that are constrained by Federal regulation to archive their audit trails (HIPAA, SOX).
But best of all, because Oracle 11g compression makes storage up to 3x cheaper, solid-state flash drives far less expensive allowing Oracle shops to forever eliminate the high costs of platter-disk I/O and enjoy data access speeds up to 300x faster.
Let's take a closer look at how one would implement Oracle 11g Data Compression in order to achieve the optimal results.
A brief history of database compression
Oracle Compression Overview
Costs and Benefits of 11g compression
Oracle11g compression syntax
Oracle 11g encrypted tablespace compression
Operational tests of the 11g data compression utility
A benchmark of 11g compression with SSD
It is expected that Oracle data compression will eventually become a default for Oracle systems, much like Oracle implemented the move from dictionary managed tablespaces to locally managed tablespaces. Eventually, this data compression may become ubiquitous, a general part of the Oracle database management engine, but its important for the Oracle database administrator to understand the ramifications of the data compression and have the ability to turn-off compression at-will.
For example, super small servers (read PC's), may not possess enough horsepower to absorb the small (but measurable) overhead of the compress/decompress routines. Remember, there is always a tradeoff between these costs vs. the saving on disk storage and allowing for information to be retrieved with the minimum amount of physical disk I/O.
A history of database compression
Data compression algorithms (such as the Huffman algorithm), have been around for nearly a century, but only today are they being put to use within mainstream information systems processing. All of the industrial strength database offer some for of data compression (Oracle, DB2, CA-IDMS), while they are unknown within simple data engines such as Microsoft Access and SQL Server. There are several places where data can be compressed, either external to the database, or internally, within the DBMS software:
Physical database compression
Hardware assisted compression - IMS, the first commercially available database offers Hardware Assisted Data Compression (HDC) which interfaces with the 3380 DASD to compress IMS blocks at the hardware level, completely transparent to the database engine.
Block/page level compression - Historical database compression uses external mechanisms that are invisible to the database. As block are written from the database, user exits invoke compression routines to store the compressed block on disk. Examples include CLEMCOMP, Presspack and InfoPak. In 1993, the popular DB2 offers built-in tablespace level compression using a COMPRESS DDL keyword.
Logical database compression
Internal Database compression operates within the DBMS software, and write pre-compressed block directly to DASD:
Table/segment-level compression - A database administrator has always had the ability to remove excess empty space with table blocks by using Oracle Data Pump or Oracle's online reorganization utility (dbms_redefinition). By adjusting storage parameters, the DBA can tightly pack rows onto data blocks. In 2003, Oracle9i release 2 introduced a table-level compression utility using a table DDL COMPRESS keyword, and here is Oracle's TPC-H compression benchmark from 2006 and this blogger noted problems with the Oracle 9i compression:
"Secondly, we implemented data segment compression, and now we have to keep running mods against the sys.obj$ table to prevent "block too fragmented to build bitmap index" errors."
Row level compression - In 2006, DB2 extended their page-level compression with row-level compression. Oracle 11g offers a row-level compression routine.
Transparent data encryption (TDE) - See here for Oracle transparent data encryption Tips
Let's take a closer look at the historical evolution of database compression.
External database compression
Legacy mainframe databases such as IDMS and DB2 allowed the DBA to choose any data compression algorithm they desired. One popular database compression program was offered by Clemson University (still a leader in data compression technologies), and their compression programs were very popular in the early 1980's with their CLEMCOMP and CLEMDCOM programs. In IDMS, a user exit in the DMCL allowed for the data compression routine to be invoked "before put" (writes), and "after get" (reads). Internally, all database compression routines try to avoid changing their internal software and rely on user exits to compress the data outbound and decompress the incoming data before it enters the database buffers. In general, data compression techniques follows this sequence:
Read compressed data (decompress):
1 - The database determines that a physical read is desired and issues an I/O request.
2 - Upon receipt of the data block from disk, Oracle un-compresses the data. This happens in RAM, very quickly.
3 - The uncompressed data block is moved into the Oracle buffer.
Write an compressed Oracle block (compress):
1 - The database determines that a physical write is desired and issues an I/O request.
2 - The database reads the data block from the buffer and calls a compression routine to quickly uncompresses the data. This happens in RAM, very quickly.
3 - The compressed data block is written to disk.
Of course, database compression has evolved dramatically since it was first introduced in the early 1980's, and Oracle 11g claims to have one of the best database compression utilities ever made. Let's take a closer look.
Oracle Compression Overview
Over the past decade Oracle introduced several types of compression so we must be careful to distinguish between the disparate tools. The 11g data compression is threshold-based and allows Oracle to honor the freelist unlink threshold (PCTFREE) for the compress rows, thereby allowing more rows per data block.
Simple index compression in Oracle 8i
Table-level compression in Oracle9ir2 (create table mytab COMPRESS)
LOB compression (utl_compress) in Oracle 10g
Row-level compression in Oracle 11g, even for materialized views (create table mytab COMPRESS FOR ALL OPERATIONS; )
The historical external compression (blocks are compressed outbound and uncompressed before presenting to the database) are far simpler because all index objects are treated the same way, whereas with the 11g table compression, a data block may contain both compressed and uncompressed row data.
For the official details on Oracle 11g table data compression, see the Oracle 11g concepts documentation. Oracle says that their software will perform a native disk read on the data block, only decompressing the row data after the physical I/O has been completed.
Within the data buffers, the fully uncompressed version of the data remains, even though the information remains compressed on the disk data blocks themselves. This leads to a discrepancy between the size of information on the data blocks and the size of the information within the data buffers. Upon applying Oracle data compression, people will find that far more rows will fit on a data block of a given size, but there is still no impact on the data base management system from the point of view of the SGA (system global area).
Because the decompression routine is called upon block fetch, the Oracle data buffers remain largely unchanged while the data blocks themselves tend to have a lot more data on them. This Oracle 11g data compression whitepaper describes the data compression algorithm:
Compress
โทรตอนนี้: 252-767-6166 เว็บไซต์ค้นหา BC Oracle บ้าน อีเมลหาเรา บทความของ oracle ฝึกอบรม oracle เคล็ดลับของ oracle Oracle Forum แค็ตตาล็อกประเภท DBA ที่ระยะไกล ปรับแต่ง oracle ฉุกเฉิน 911 สนับสนุน RAC สนับสนุนแอพพลิเค วิเคราะห์ การออกแบบ นำไปใช้ สนับสนุนของ oracle ปรับ SQL รักษาความปลอดภัย Oracle UNIX Oracle Linux การตรวจสอบ สนับสนุนระยะไกล แผนการระยะไกล บริการระยะไกล เซิร์ฟเวอร์แอพลิเคชัน โปรแกรมประยุกต์ ฟอร์มของ oracle เว็บไซต์ oracle ปรับรุ่นโปรแกรมประยุกต์ เซิร์ฟเวอร์ SQL แนวคิดของ oracle ซอฟต์แวร์สนับสนุน สนับสนุนระยะไกล พัฒนา นำไปใช้ บริการให้คำปรึกษา ให้คำปรึกษาราคา ต้องการความช่วยเหลือ โปสเตอร์ของ oracle หนังสือ oracle สคริปต์ของ oracle ไอออน ฐานข้อมูล Excel บล็อก Burleson ดอน สำหรับผู้ดูแลฐานข้อมูล oracle 11g เคล็ดลับการบีบอัดข้อมูลเคล็ดลับ 11g oracle Burleson ให้คำปรึกษา เหล่านี้เป็นการตัดตอนจากหนังสือ "Oracle g 11 คุณลักษณะใหม่" การทดสอบแสดงว่า ผลรวม 11g เป็นอัตราความเร็วช้าของธุรกรรม แต่สร้างเขียนน้อยลงเนื่องจากความหนาแน่นแถวสูงบนบล็อกข้อมูล ดูนี้มาตรฐานของการเข้ารหัสลับข้อมูลที่โปร่งใส ยัง ดูบันทึกนี้ด่วนวิธีการบีบพื้นที่ตาราง Oracleจริงอยู่ที่ว่า ราคาจัดเก็บข้อมูล (ดิสก์) ได้ลดลงอย่างมากในทศวรรษ (และยังคงลดลงอย่างรวดเร็ว), บีบอัดข้อมูล Oracle มีประโยชน์น่าสนใจมากขึ้นกว่าเพียงแค่บันทึกบนดิสก์จัดเก็บข้อมูลต้นทุน เนื่องจากข้อมูลนั้นสามารถบีบสูง สามารถนำข้อมูลออกจากอุปกรณ์ดิสก์กับ IO น้อยจริง ซึ่งก็ปรับปรุงประสิทธิภาพภายใต้เงื่อนไขบางประการ โปรดทราบว่า มีความเข้าใจผิดทั่วไปบีบอัดตาราง 11g คลายการบีบอัดข้อมูลในขณะที่อ่าน และจัดเก็บในแบบบีบอัดในแคช ที่ถูกต้องไม่มากเนื่องจากคุณสมบัติเด่นของการบีบอัดฐานข้อมูลอย่างใดอย่างหนึ่งที่เราไม่มีการบีบอัดข้อมูลก่อนอ่านเรื่องและข้อมูลอยู่ในแบบบีบอัดในแคช ดัง การบีบอัดตาราง Oracle ไม่เพียงช่วยลูกค้าประหยัดพื้นที่ดิสก์ ยังช่วยเพิ่มประสิทธิภาพของแคชเนื่องจากบล็อกเพิ่มเติมสามารถบันทึกลงในหน่วยความจำขณะนี้ ตาม whitepaper Oracle ในการบีบอัดข้อมูล 11 g, CPU ค่าใช้จ่ายสำหรับการดำเนินงานการบีบอัด/ขยายจะน้อยที่สุด สำคัญ Oracle11g รวมข้อมูลจะได้จากในร้านที่จำกัดตามข้อบังคับของรัฐบาลกลางเพื่อเก็บบันทึกการตรวจสอบของพวกเขา (HIPAA ท่าน) แต่ดีที่สุดทั้งหมด เพราะอัด 11g Oracle ทำให้จัดเก็บได้ถึง 3 x ถูกกว่า โซลิดสเตตแฟลชไดรฟ์ไกลแพงให้ Oracle เพื่อตลอดไปกำจัดต้นทุนที่สูงของ I/O ของดิสก์แผ่นเสียง และเพลิดเพลินกับการเข้าถึงข้อมูลความเร็ว x เร็วถึง 300ลองมาดูใกล้ชิดในวิธีหนึ่งจะใช้ Oracle 11g บีบอัดข้อมูลเพื่อให้บรรลุผลลัพธ์ดีที่สุดประวัติโดยย่อของการบีบอัดฐานข้อมูล ภาพรวมรวม oracle ต้นทุนและผลประโยชน์ของ 11g รวม Oracle11g รวมไวยากรณ์บีบพื้นที่ตาราง 11g เข้า oracle ทดสอบการดำเนินงานของโปรแกรมอรรถประโยชน์การบีบอัดข้อมูล 11g มาตรฐานของ 11g รวมกับ SSDที่คาดหวังว่า การบีบอัดข้อมูล Oracle ก็จะกลายเป็น ค่าเริ่มต้นสำหรับระบบ Oracle มากเช่น Oracle ดำเนินไปจากพจนานุกรมจัดการพื้นที่การจัดการภายในพื้นที่ ในที่สุด การบีบอัดข้อมูลนี้อาจแพร่หลาย ส่วนทั่วไปของ Oracle ฐานข้อมูลการจัดการเครื่องยนต์ แต่มันสำคัญสำหรับผู้ดูแลระบบฐานข้อมูล Oracle และหันออกบีบอัดที่จะสามารถเข้าใจ ramifications ของการบีบอัดข้อมูลได้ ตัวอย่าง เซิร์ฟเวอร์ขนาดเล็กพิเศษ (อ่านเครื่องพีซี), อาจไม่มีแรงม้าเพียงพอซับค่าโสหุ้ย (แต่วัด) ของคำสั่งบีบอัด/ขยาย จำ มีเสมอเป็นข้อดีข้อเสียระหว่างต้นทุนเหล่านี้เทียบกับบันทึกบนดิสก์จัดเก็บข้อมูลและให้ข้อมูลเพื่อดึงข้อมูล ด้วยยอดเงินต่ำสุดของ I/O ที่ดิสก์ทางกายภาพ ประวัติของการบีบอัดฐานข้อมูล Data compression algorithms (such as the Huffman algorithm), have been around for nearly a century, but only today are they being put to use within mainstream information systems processing. All of the industrial strength database offer some for of data compression (Oracle, DB2, CA-IDMS), while they are unknown within simple data engines such as Microsoft Access and SQL Server. There are several places where data can be compressed, either external to the database, or internally, within the DBMS software: Physical database compressionHardware assisted compression - IMS, the first commercially available database offers Hardware Assisted Data Compression (HDC) which interfaces with the 3380 DASD to compress IMS blocks at the hardware level, completely transparent to the database engine. Block/page level compression - Historical database compression uses external mechanisms that are invisible to the database. As block are written from the database, user exits invoke compression routines to store the compressed block on disk. Examples include CLEMCOMP, Presspack and InfoPak. In 1993, the popular DB2 offers built-in tablespace level compression using a COMPRESS DDL keyword.Logical database compression Internal Database compression operates within the DBMS software, and write pre-compressed block directly to DASD:Table/segment-level compression - A database administrator has always had the ability to remove excess empty space with table blocks by using Oracle Data Pump or Oracle's online reorganization utility (dbms_redefinition). By adjusting storage parameters, the DBA can tightly pack rows onto data blocks. In 2003, Oracle9i release 2 introduced a table-level compression utility using a table DDL COMPRESS keyword, and here is Oracle's TPC-H compression benchmark from 2006 and this blogger noted problems with the Oracle 9i compression:"Secondly, we implemented data segment compression, and now we have to keep running mods against the sys.obj$ table to prevent "block too fragmented to build bitmap index" errors." Row level compression - In 2006, DB2 extended their page-level compression with row-level compression. Oracle 11g offers a row-level compression routine.Transparent data encryption (TDE) - See here for Oracle transparent data encryption TipsLet's take a closer look at the historical evolution of database compression. External database compression Legacy mainframe databases such as IDMS and DB2 allowed the DBA to choose any data compression algorithm they desired. One popular database compression program was offered by Clemson University (still a leader in data compression technologies), and their compression programs were very popular in the early 1980's with their CLEMCOMP and CLEMDCOM programs. In IDMS, a user exit in the DMCL allowed for the data compression routine to be invoked "before put" (writes), and "after get" (reads). Internally, all database compression routines try to avoid changing their internal software and rely on user exits to compress the data outbound and decompress the incoming data before it enters the database buffers. In general, data compression techniques follows this sequence:Read compressed data (decompress):1 - The database determines that a physical read is desired and issues an I/O request. 2 - Upon receipt of the data block from disk, Oracle un-compresses the data. This happens in RAM, very quickly. 3 - The uncompressed data block is moved into the Oracle buffer.Write an compressed Oracle block (compress):1 - The database determines that a physical write is desired and issues an I/O request. 2 - The database reads the data block from the buffer and calls a compression routine to quickly uncompresses the data. This happens in RAM, very quickly. 3 - The compressed data block is written to disk.Of course, database compression has evolved dramatically since it was first introduced in the early 1980's, and Oracle 11g claims to have one of the best database compression utilities ever made. Let's take a closer look.
Oracle Compression Overview
Over the past decade Oracle introduced several types of compression so we must be careful to distinguish between the disparate tools. The 11g data compression is threshold-based and allows Oracle to honor the freelist unlink threshold (PCTFREE) for the compress rows, thereby allowing more rows per data block.
Simple index compression in Oracle 8i
Table-level compression in Oracle9ir2 (create table mytab COMPRESS)
LOB compression (utl_compress) in Oracle 10g
Row-level compression in Oracle 11g, even for materialized views (create table mytab COMPRESS FOR ALL OPERATIONS; )
The historical external compression (blocks are compressed outbound and uncompressed before presenting to the database) are far simpler because all index objects are treated the same way, whereas with the 11g table compression, a data block may contain both compressed and uncompressed row data.
For the official details on Oracle 11g table data compression, see the Oracle 11g concepts documentation. Oracle says that their software will perform a native disk read on the data block, only decompressing the row data after the physical I/O has been completed.
Within the data buffers, the fully uncompressed version of the data remains, even though the information remains compressed on the disk data blocks themselves. This leads to a discrepancy between the size of information on the data blocks and the size of the information within the data buffers. Upon applying Oracle data compression, people will find that far more rows will fit on a data block of a given size, but there is still no impact on the data base management system from the point of view of the SGA (system global area).
Because the decompression routine is called upon block fetch, the Oracle data buffers remain largely unchanged while the data blocks themselves tend to have a lot more data on them. This Oracle 11g data compression whitepaper describes the data compression algorithm:
Compress
การแปล กรุณารอสักครู่..
