Showing posts with label T-SQL. Show all posts
Showing posts with label T-SQL. Show all posts

Wednesday, 7 October 2020

Updating massive amount of rows whilst avoiding blocking

The following SQL is a good means to split an update on a massive table into smaller chunks, whilst reducing blocking. The method is to update top N rows from a table, within a while loop, with a wait between loops, to avoid excessive blocking of the table and allow other transactions between each loop. Further blocking reduction is discussed in the link below.

set nocount on

declare 
    @ChunkSize              int = 1000,                         -- count rows to remove in 1 chunk 
    @TimeBetweenChunks      char(8) = '00:00:01',               -- interval between chunks
    
    @Start                  datetime,
    @End                    datetime,
    @Diff                   int,
    
    @MessageText            varchar(500),
    
    @counter                int = 1,
    @RowCount               int = 1,
    @TotalRowsToUpdate      bigint,
    @TotalRowsLeft          bigint
    


-- total row count to update
set @TotalRowsToUpdate = (select count(*)
                            from [Table1]
                                join [Table2] on
                                    btid = tBtID
                            where   btStatusID = 81)


set @TotalRowsLeft = @TotalRowsToUpdate
set @MessageText = 'Total Rows to Update = ' + cast(@TotalRowsLeft as varchar) raiserror (@MessageText,0,1) with nowait
print ''


-- begin cycle
while @RowCount > 0 begin

    set @Start = getdate()

    -- update packages
    update top (@ChunkSize) bti
        set btstatusid = 154,
            btType = 1
    from [Table1] bti
        join [Table2] on
            btid = tBtID
    where   btStatusID = 81
    

    set @RowCount = @@ROWCOUNT

    -- measure time
    set @End = getdate()
    set @Diff = datediff(ms,@Start,@End)

    set @TotalRowsLeft = @TotalRowsLeft - @RowCount
    set @MessageText = cast(@counter as varchar) + ' - Updated ' + cast(@RowCount as varchar) + ' rows in ' + cast(@Diff as varchar) + ' milliseconds - total ' + cast(@TotalRowsLeft as varchar) + ' rows left...'

    -- print progress message
    raiserror (@MessageText,0,1) with nowait


    set @counter += 1

    WAITFOR DELAY @TimeBetweenChunks

end

Resources:

Stack Overflow post on this subject (where SQL above was taken from): https://dba.stackexchange.com/questions/276314/how-to-avoid-table-lock-escalation

Wednesday, 9 November 2016

Creating Unique 10 Character Strings

A nice way to make a unique 10 character string is to generate a GUID, strip out the hypehens, and take only the left / right 10 characters as follows:
select right(replace(CONVERT(varchar(255), NEWID()), '-',''),10)
 This produces no duplicates over 1 million rows, when tested as follows:
create table #tmpID(ID varchar(10))

declare @i int
set @i = 1
while @i <1000000
begin
       insert into #tmpID
       select right(replace(CONVERT(varchar(255), NEWID()), '-',''),10)
       set @i = @i+1
end

select ID, count(*) [No of IDs]
from #tmpID
group by ID
having count(*) >1

Monday, 7 November 2016

Profiler - Querying Trace Files From SSMS

If you have trace data that has been saved to a file, it is possible to query this data via SQL Server Management Studio, and run SQL commands against the file. Example SQL:

SELECT *
FROM ::fn_trace_gettable('c:\TraceFile.trc', default)
This makes it much easier to work with the trace file comapred to opening the file within SQL Server Profiler. It means it is also possible to sort, filter, apply aggregations (sums, counts, etc) against the data. Of course, you could always save the trace data to a table from within SQL Server Profiler, and then perform these sorts of SQL queries against that table, but using fn_trace_gettable, you don't need to.

Wednesday, 3 February 2016

Estimate time remaining for backup / restores

Use the following SQL to estimate time remaining for active backup / restore jobs within SQL Server:

select    session_id,
    start_time,
    status,
    command,
    percent_complete,
    estimated_completion_time,
    estimated_completion_time / 60 / 1000 as estimate_completion_minutes,    dateadd(n,(estimated_completion_time / 60 / 1000),getdate()) as estimated_completion_timefrom    sys.dm_exec_requests
where    command = 'BACKUP DATABASE'
    or command = 'RESTORE DATABASE'

Monday, 3 August 2015

Function to Proper Case a text string

The following function will capitalise the first letter of each word:
create function ProperCase(@Text as varchar(8000))
returns varchar(8000)
as
begin
   declare @Reset bit;
   declare @Ret varchar(8000);
   declare @i int;
   declare @c char(1);

   select @Reset = 1, @i=1, @Ret = '';

   while (@i <= len(@Text))
    select @c= substring(@Text,@i,1),
               @Ret = @Ret + case when @Reset=1 then UPPER(@c) else LOWER(@c) end,
               @Reset = case when @c like '[a-zA-Z]' then 0 else 1 end,
               @i = @i +1
   return @Ret
end
Credit to the Stack Overflow post here: http://stackoverflow.com/questions/230138/sql-server-make-all-upper-case-to-proper-case-title-case

Wednesday, 27 May 2015

Listing Stored Procedures used by SSRS

The following SQL will allow you to list all stored procedures used by Reporting Services reports. Run this against the ReportServer database on the SQL Server your Reporting Services installation uses:

;with xmlnamespaces
(
default
'http://schemas.microsoft.com/sqlserver/reporting/2008/01/reportdefinition',
'http://schemas.microsoft.com/SQLServer/reporting/reportdesigner' AS rd
)

select
       name
       ,x.value('CommandType[1]', 'VARCHAR(50)') AS CommandType
       ,x.value('CommandText[1]','VARCHAR(50)') AS CommandText

from (

       select
              name
              , cast(cast(content AS VARBINARY(MAX))as xml) as reportXML
       from
              ReportServer.dbo.Catalog
       where
              Name not like '%.gif'
              and Name not like '%.jpg'
              and Name not like '%.jpeg'

) a
cross apply reportXML.nodes('/Report/DataSets/DataSet/Query') r(x)
where

       x.value('CommandType[1]', 'VARCHAR(50)') = 'StoredProcedure'


Note, the 4th line, it may be necessary to change "2008" to "2003", "2005", "2010" depending on the version(s) of Visual Studio / Report Builder / BIDS, SSDT you've used to create your reports.

Credit to Jacob Sebastian, I used his post as a basis here:
http://beyondrelational.com/modules/2/blogs/28/posts/10446/how-to-find-all-stored-procedures-used-by-report-server.aspx

Sunday, 3 May 2015

Transaction Isolation Levels

Overview of transaction isolation levels, from least to most restorictive (i.e. least chance of causing locks, to most likely to cause locks):

SNAPSHOT:
This transaction level takes a copy of the data being read and puts it into tempdb. This ensures that the results at the start and end of a transaction will be identical, as it refers to a copy of the data, rather than putting locks on the actual data. 

READ UNCOMMITTED:
This is the least restrictive transaction isolation level and the least likely to result in locks. The READ UNCOMMITTED transaction level allows transactions to make modifications to data read by the current transaction. 

Use this level if you do not want the Database Engine to issue shared locks to prevent other transactions from making modifications to data read by the current transaction. READ UNCOMMITTED is the least restrictive transaction isolation level.

READ UNCOMMITTED does not prevent other transactions from making modifications to data read by the current transaction

READ COMMITTED:
This transaction level blocks statements from reading data that has been modified by but not committed by other transactions. At this transaction level, data can be changed by other transactions between the execution of individual statements within the current transaction.

If you set the READ COMMITTED transaction levels, statements cannot read data that has been modified by other transactions that are uncommitted. This prevents dirty reads. Data can be changed by other transactions between the execution of individual statements within the current transaction. READ COMMITTED is the SQL Server 2012 default transaction isolation level.

READ COMMITTED transaction isolation level blocks other statements from reading data that has been modified but not committed by other transactions, but data can be changed by other transactions between the execution of individual statements within the current transaction.

REPEATABLE READ:
This transaction level blocks statements from reading data that has been modified but not committed. When this level is set, transactions cannot alter data that has been read by the current transaction until that transaction completes. REPEATABLE READ uses shared locks on all data read by each statement until the transaction completes.

If you set the REPEATABLE READ transaction level, statements are unable to read data that has been modified by uncommitted. No transactions can alter data that has been read by the current transaction until that transaction completes. REPEATABLE READ uses shared locks on all data read by each statement until the transaction completes.

REPEATABLE READ transaction isolation level is set, no transactions can alter data that has been read by the current transaction until that transaction completes

SERIALIZABLE:
The SERIALIZABLE transaction level is the most restrictive transaction isolation level and likely to result in more locks than other transaction isolation levels. When this isolation level is set, statements cannot read data that has been modified but not yet committed by other transactions, transactions are unable to modify data that has been read by the current transaction until that transaction completes, and other transactions are unable to insert new rows into the range of rows being read by the current transaction.

If you set the SERIALIZABLE transaction level, statements cannot read data that has been modified but not yet committed by other transactions; transactions are unable to modify data that has been read by the current transaction until that transaction completes; other transactions are unable to insert new rows into the range of rows being read by the current transaction. The SERIALIZABLE transaction level is the most restrictive isolation level.

Statements cannot read data that has been modified but not yet committed by other transactions.
Other transactions are unable to modify data that has been read by the current transaction until the current transaction completes.

Other transactions are unable to insert new rows into the range of rows being read by the current transaction.

Concurrency issues with each transaction isolation level:

This post gives some great overview of transaction isolation levels. The following table and definitions of dirty / repeatable / phantom reads are taken from there: 


Isolation level
Dirty read
Nonrepeatable read
Phantom read
Read uncommitted
Read committed
Repeatable read
Serializable
Snapshot

  • Dirty read: Session 1 begins a transaction and modifies data. Session 2 reads the modified data before Session 1 commits the transaction. Session 2 is reading data that exists only in theory because the Session 1 transaction might be rolled back. If that occurs, the data read by Session 2 is no longer valid.
  • Nonrepeatable read: Session 1 begins a transaction and retrieves a row of data from a table. Session 2 updates that row. Session 1 tries to retrieve the row once more, still within the original transaction, but receives different results because of the Session 2 update.
  • Phantom read: Session 1 begins a transaction and retrieves several rows of data from a table. Session 2 inserts a row into the same table, and the insert coincides with the search criteria of the Session 1 statement. When Session 1 reruns the query, the new row added by Session 2 is now included in the results.

Trace Flags

Trace flags are used within SQL Server to provide information / logging about specific system events, or to change specific configuration of the SQL Server instance. 

Trace flags can be set on / off for the current connection only, using the following T-SQL code:
-- Turn on trace flag 1234
DBCC TRACEON (1234);
GO

-- Turn off trace flag 1234
DBCC TRACEOFF (1234);
GO
Traces can be set on / off globally (i.e. for ALL connections to the SQL Engine) with the addition of the -1 argument at the end:
 -- Turn on trace flag 1234, globally
DBCC TRACEON (1234, -1);
GO
Multiple traces can be switched on by listing them within the parenthesis:
-- Turn on trace flags 1234, 2345 and 3456
DBCC TRACEON (1234, 2345, 3456);
GO
You can see which trace flags are currently active as follows (if none are active, a message of "DBCC execution completed. If DBCC printed error messages, contact your system administrator" will appear): 
-- Check status of current trace flags
DBCC TRACESTATUS
Only members of the sysadmin fixed group can switch traces on / off.

Useful Trace Flags:
1222: Examine deadlocks
1204: Write deadlock information to SQL Server log

Trace flag resources:
MSDN: https://msdn.microsoft.com/en-us/library/ms188396.aspx
SQL Mag: http://sqlmag.com/sql-server/investigating-trace-flags
Stack Exchange: http://dba.stackexchange.com/questions/27704/sql-server-concise-list-of-trace-flags

Saturday, 15 February 2014

Adding compression to a table at row or page level

It is possible to add compression to a table at row or page level. At row level, any row values that can be stored in a smaller data type, will be. At page level, row level compression is used, and any duplicate data is rolled up within the page, creating more space in each page. The gain in disk space (and therefore fewer reads / writes) is to the detriment of CPU utilisation when reading / writing the data. You can check how much space you can save by using row or page compression by using the following stored procedure:
EXEC sp_estimate_data_compression_savings [Schema], [TableName], NULL, NULL, 'ROW'
You can then add the desired compression type to a table as follows:
ALTER TABLE [TableName]
REBUILD WITH (DATA_COMPRESSION = PAGE)

Tuesday, 21 January 2014

Kill all current user processes connecting to database

Sometimes you may wish to sever any user connections to a database before performing some sort of action. One way to do this, is as follows, by getting a list of all SPIDs relating to a domain user, and appending KILL to the start. Run the following SQL, then copy, paste and run the output:
select 'Kill ' + cast(spid as varchar) from sys.sysprocesses
where loginame like 'DOMAIN\%'
Alternatively, as suggested by Pinal Dave in his blog post, you could set the server to Single User Mode and back to Multi User Mode as follows. This method is very quick and neat, however, this will kill ALL connections to the database, not just those from domain users:
USE master;
GO
ALTER DATABASE AdventureWorks
SET SINGLE_USER
WITH ROLLBACK IMMEDIATE;
ALTER DATABASE AdventureWorks
SET MULTI_USER;
GO
Credit for Pinal Dave blog post: http://blog.sqlauthority.com/2009/02/06/sql-server-quickest-way-to-kill-all-threads-kill-all-user-session-kill-all-processes/

Wednesday, 8 January 2014

Find last time a table has been accessed

The following SQL script will give you information about the last time a table has been accessed, or updated:
select
      so.name,
      last_user_seek = max(last_user_seek),
      last_user_scan = max(last_user_scan),
      last_user_lookup = max(last_user_lookup),
      last_user_update = max(last_user_update)
from
      sys.dm_db_index_usage_stats s
inner join sysobjects so
      on s.object_id = so.id
where
      [database_id] = db_id()
group by
      so.name

Updating massive amount of rows whilst avoiding blocking

The following SQL is a good means to split an update on a massive table into smaller chunks, whilst reducing blocking. The method is to upda...