Working with Very Large Tables
Like a Pro in SQL Server 2014
Guy Glantser
Image courtesy of marin / FreeDigitalPhotos.net
Copyright © 2014 Madeira Ltd
All Rights Reserved
Full rights, including copyrights, belong exclusively to Madeira Ltd
No use of the materials, in any form, is allowed,
unless receiving a prior written permission from Madeira Ltd
Name:
Email Address:
Twitter:
Blog:
Podcast:
Guy Glantser
[email protected]
@guy_glantser
www.madeirasql.com/author/guyglantser
www.sqlserverradio.com
Image courtesy of Mister GC / FreeDigitalPhotos.net
Working with Very Large Tables
Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net
Working with Very Large Tables
Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net
Working with Very Large Tables
Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net
Working with Very Large Tables
Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net
Working with Very Large Tables
Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net
5GB
5TB
Images courtesy of satit_srihin & John Kasawa / FreeDigitalPhotos.net
It depends…
Image courtesy of David Castillo Dominici / FreeDigitalPhotos.net
A Very Large Table
is One that Requires
Special Care
Image courtesy of stockimages / FreeDigitalPhotos.net
File.Events
Web.PageViews
• Query the Table
Image courtesy of Master isolated images / FreeDigitalPhotos.net
• Query the Table
• Update the Table
Image courtesy of Stuart Miles / FreeDigitalPhotos.net
• Query the Table
• Update the Table
• Manage the Table
Image courtesy of Stuart Miles / FreeDigitalPhotos.net
•
•
•
•
Query the Table
Update the Table
Manage the Table
Maintain the Table
Image courtesy of xedos4 / FreeDigitalPhotos.net
• Partition the Table
• Archive/Delete
Old Partitions
• Spread Partitions
Between Storage Tiers
Image courtesy of farconville / FreeDigitalPhotos.net
• Compress the Data
• Bulk Insert Rows
• Optimize Parallelism
Image courtesy of farconville / FreeDigitalPhotos.net
• Use a Lower
Isolation Level
• In-Memory OLTP
• Columnstore Indexes
Image courtesy of farconville / FreeDigitalPhotos.net
Image courtesy of Master isolated images / FreeDigitalPhotos.net
The Ascending Key
Problem
Image courtesy of suphakit73 / FreeDigitalPhotos.net
•
•
•
•
•
Update Statistics Manually
Trace Flag 2371
New Cardinality Estimator
Filtered Statistics
Incremental Statistics
Image courtesy of suphakit73 / FreeDigitalPhotos.net
Image courtesy of Stuart Miles / FreeDigitalPhotos.net
The WRITELOG
Wait Type
Image courtesy of stockimages / FreeDigitalPhotos.net
Delayed
Durability
Image courtesy of stockimages / FreeDigitalPhotos.net
Transaction Properties (ACID):
• Atomicity
• Consistency
• Isolation
• Durability
Checkpoint
Buffer Pool
Data File
Request
Transaction Log
Buffer
WRITELOG
Log File
Checkpoint
Buffer Pool
Data File
Request
Transaction Log
Buffer
Log File
Can be controlled by:
• Database (Disabled | Allowed |
Forced)
• Transaction (Commit)
• Atomic Block
(Natively Compiled Procedure)
Image courtesy of digitalart / FreeDigitalPhotos.net
The log buffer is flushed to disk when:
• A fully durable transaction
in the same database
makes a change in the database
and successfully commits
Image courtesy of Keerati / FreeDigitalPhotos.net
The log buffer is flushed to disk when:
• The user executes the system
stored procedure
“sys.sp_flush_log” successfully
Image courtesy of Keerati / FreeDigitalPhotos.net
The log buffer is flushed to disk when:
• The in-memory transaction log
buffer fills up and automatically
flushes to disk
Image courtesy of Keerati / FreeDigitalPhotos.net
The log buffer is flushed to disk when:
• Every few milliseconds
Image courtesy of Keerati / FreeDigitalPhotos.net
The following transactions are
always fully durable:
• Transactions that perform
write operations against tables
that are enabled for either
Change Tracking or Change Data Capture
Image courtesy of Iamnee / FreeDigitalPhotos.net
The following transactions are
always fully durable:
• Cross-database transactions
or distributed transactions
Image courtesy of Iamnee / FreeDigitalPhotos.net
The following features apply only to
transactions that have been made
durable:
• Transaction Replication –
Transactions are only replicated
after they have been made durable
Image courtesy of Stuart Miles / FreeDigitalPhotos.net
The following features apply only to
transactions that have been made
durable:
• Log Backup – Only transactions
that have been made durable
are included in the backup
Image courtesy of Stuart Miles / FreeDigitalPhotos.net
The following features apply only to
transactions that have been made
durable:
• Log Shipping – Only transactions
that have been made durable
are shipped to the secondaries
Image courtesy of Stuart Miles / FreeDigitalPhotos.net
What are the benefits?
• Less roundtrips to the log file on disk
• Less WRITELOG waits
• Shorter transactions
• Locks are released earlier
• Better throughput
Image courtesy of Ambro / FreeDigitalPhotos.net
But remember that
you might lose data!
Image courtesy of xedos4 / FreeDigitalPhotos.net
Index Rebuild
Image courtesy of suphakit73 / FreeDigitalPhotos.net
Before SQL Server 2014 we could:
• Rebuild all partitions offline
• Rebuild all partitions online
• Rebuild a specific partition offline
Image courtesy of David Castillo Dominici / FreeDigitalPhotos.net
In SQL Server 2014 we can:
• Rebuild a specific partition
online
• Wait at low priority when:
• Rebuilding an index online
• Switching partitions
Image courtesy of David Castillo Dominici / FreeDigitalPhotos.net
• Wait in a separate low priority queue
• Wait for a specified
number of minutes
Image courtesy of ddpavumba / FreeDigitalPhotos.net
• After that, choose one of 3 options:
• Continue to wait at normal priority
• Abort the operation
• Kill all blockers
and start the operation
Image courtesy of ddpavumba / FreeDigitalPhotos.net
“Normal”
Queue
Running
Online
Insert
Index Rebuild
Image courtesy of artur84 / FreeDigitalPhotos.net
“Normal”
Queue
Running
Online
Insert
Index Rebuild
Low Priority
Queue
Image courtesy of artur84 / FreeDigitalPhotos.net
Working with Very Large Tables
Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net
Working with Very Large Tables
Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net
A Very Large Table
is One that Requires
Special Care
Image courtesy of stockimages / FreeDigitalPhotos.net
Working with Very Large Tables
Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net
• Query the Table
• Update the Table
• Manage the Table
• Maintain the Table
Image courtesy of Master isolated images, Stuart Miles & xedos4 / FreeDigitalPhotos.net
Working with Very Large Tables
Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net
Solving the Ascending Key problem
• Update Statistics Manually
• Trace Flag 2371
• New Cardinality Estimator
• Filtered Statistics
• Incremental Statistics
Image courtesy of suphakit73 / FreeDigitalPhotos.net
Solving the WRITELOG problem
with Delayed Durability
• Less roundtrips to the log file on disk
• Less WRITELOG waits
• Shorter transactions & Locks
• Better throughput
Image courtesy of Ambro / FreeDigitalPhotos.net
Solving the Index Rebuild problem
• Rebuild a specific partition online
• Wait at low priority
Image courtesy of suphakit73 / FreeDigitalPhotos.net
Working with Very Large Tables
Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net
New Features in SQL Server 2014
• New Cardinality Estimator
• Incremental Statistics
• Delayed Durability
• Online Indexing Enhancements
Name:
Email Address:
Twitter:
Blog:
Podcast:
Guy Glantser
[email protected]
@guy_glantser
www.madeirasql.com/author/guyglantser
www.sqlserverradio.com
Image courtesy of Nuttapong / FreeDigitalPhotos.net
Image courtesy of David Castillo Dominici / FreeDigitalPhotos.net