Sunday, February 12, 2012

Bitwise NOT Operation

I was looking into some of the System stored procedures like sp_denylogin, sp_grantLogin etc. I found bit wise operations like

update master.dbo.sysxlogins set xstatus = (xstatus & ~1) | 2, xdate2 = getdate()
where name = @.loginame and srvid IS NULL

How does the bitwise NOT(~) works with different datatypes like INT, SMALLINT and TINYINT?

Look at the following sample code and result. I just wanted to understand how it works with int and smallint

declare @.a int
declare @.b smallint
declare @.c tinyint

select @.a = 1, @.b = 1, @.c = 1

select @.a=~@.a, @.b= ~@.b, @.c= ~@.c

select @.a, @.b, @.c

Result
~~~~~
-2 -2 254

Thanks in advance
GnanaInteresting question! In a former life, I worked on addressable converter interfaces, and much was done with bitmaps...so I played a bit also...

Note the results if you execute the following select in your example:
select @.a, @.b, @.c, cast(@.a AS varbinary), cast(@.b as varbinary), cast (@.c as varbinary)

Result
~~~~~
-2 -2 254 0xFFFFFFFE 0xFFFE 0xFE

Now you get to see the binary results...which still does not answer your question, but I betcha (as I head back to BOL) that the answer lies in there as the way the sign bit is interpreted in the larger two data types.

Betcha that the tinyint data type does not have a sign bit.

yep...that's it...from BOL:tinyint

Integer data from 0 through 255. Storage size is 1 byte.

So the issue is not so much that the bitwise operation actually WORKS any differently...what you are seeing is just the PRESENTATION differences between data types (the final select assumes you want the data interpreted in it's numeric value, not the bit-representation of the data in the variable...so you get a negative number because you have set the sign bit on with the bitwise operation you used on the larger data types (that have sign bits).

In other words, the bitwise operator works exactly the same on all three data types, it just LOOKS like it works differently when you select it back to see the results (without casting, that is).|||Nice explanation, Paul.|||Yeah, thanks! I learned something too (something I usually try to avoid at all costs)...though we don't use much bit manipulation here (at least not out where everyone can see ;) ).

No comments:

Post a Comment