Thursday, June 24, 2004

All you ever wanted to know about the IDisposable and was ashamed to ask

I have some coworkers who are new to the .NET world, and still need to be introduced to the platform best practices. One of the key knowledge that every .NET programmer should have is the IDisposable interface; when to implement it and when to call it. What happens when you fail to do that is one of the points of this article, but for now let's just say that it can lead to difficult to track software bugs, memory and resource leaks.
Before writing this I researched the Web a lot to find a single place that I could point my colleagues in order to learn about IDisposable, but I could not find an article that covers it both fully and deeply, so I decided to start writing.
As I don't have much free time for now, I will post this on my blog one chapter at a time, probably one topic each day, and when it is finished, I'll post this on sites like Code Project as a finished (huge) article.

The topics I intend to cover are (note that this is my initial planned outline, may it changes a little while I write, but you can consider this as a bare minimum):

The beginning
What's IDisposable?
What are the responsibilities of the IDisposable for the class user?
What happens if I don't call Dispose? Why is IDisposable needed?
Isn't .NET garbage-collected?

Calling Dispose
What happens if I call Dispose twice?
What happens if I use an object that had Dispose called?
Where's the correct place to call Dispose?

Implementing IDisposable
What is the correct way of implementing IDisposable?
Don't forget to throw ObjectDisposedException.

C# Support to IDisposable:
What about C# destructors?
The "using" statement.
The "foreach" statement.

Design implications:
Close or Dispose? Why do both exist?
Why did someone choose a private implementation of IDisposable?
Implications for library writers
Should I dispose a whole hierarchy of IDisposable instances or only the root?
Mis"using" C# support for IDisposable.

Useful links for more information

Friday, June 18, 2004

Leaks, leaks, everywhere...

I think I'm becoming an authority on finding .NET leaks: I just found a handle leak on the EventLog class.
The problem, this time is with the static method (decompiled with Lutz Roeder's .NET Reflector):

public static void WriteEntry(string source, string message, EventLogEntryType type, int eventID,
short category, byte[] rawData)
{
EventLog log1 = new EventLog();
log1.Source = source;
log1.WriteEntry(message, type, eventID, category, rawData);
}



Noticed? EventLog implements IDisposable and the code should be this:

public static void WriteEntry(string source, string message, EventLogEntryType type, int eventID,
short category, byte[] rawData)
{
using (EventLog log1 = new EventLog())
{
log1.Source = source;
log1.WriteEntry(message, type, eventID, category, rawData);
}
}



So, the lesson is: if you care about not leaking handles (e.g., in server applications, where a few handles per processing unit can shut down a whole service), don't use LogEvent static methods. Create an instance and call its methods, and dispose it yourself.

Eric is right!

This post isn't about programming, but is about making money programming. Eric is talking about the 22 Immutable Laws of Marketing, and today he posted Law #13: The Law of Sacrifice. Nothing could be more appropriate to me today.
I had this insanely good project, and that would mean a very nice amount of money. Today, I had to say "no" to it because the deadlines are too risky. More than this, it would mean a stop on the development a lot of nice features on Crivo that would fit so well to so many customers, that, in the long term would mean that we're hurting Crivo's feature.
But it's hard to say "no" and it's hard to keep this decision after saying that...

Wednesday, June 16, 2004

Deriving from Panel woes

If you're deriving from Panel, set AutoScroll to true and set AutoScrollMinSize to some big numbers, you get a Panel with ScrollBars, right?
Right: everything works, but the OnMouseWheel event doesn't happen, and the mouse wheel don't work. After googling a lot, I happened to discover that the mouse wheel is only sent to controls when they have the focus (doh! it should be obvious!).
So, correcting this is a simple matter of SetStyle(ControlStyles.Selectable, true) and then, at some place (probably on OnMouseDown or OnMouseEnter), you call Focus().
Voilá! Instant mousewheel support on your Panel!

I'm new to this blog thing...

And I'm yet discovering new things. By default, Blogger has the title field disabled, and the blog started to get confusing. I've changed the setting, but if you subscribed to my Atom feed, you probably noticed that the older posts became duplicated, because Blogger was sending a bogus title for previous posts...

Tuesday, June 15, 2004

More on Try/Finally.

Along the lines of my previous post, this month's MSDN Magazine has an interesting article about unexpected errors in managed applications. On the "Some Best Practices" topic, Jason puts some quite strong arguments that go exactly along the lines of my post.
Some highlights:

* Don't catch or throw base exception types
* Use finally blocks liberally

(...)

Now let's look at the guideline surrounding finally blocks. In general, well-written managed code contains more try/finally constructs than it does try/catch constructs. The reason for this is that finally blocks guarantee the ability to perform method-level clean-up operations in both exceptional and non-exceptional executions. Many methods include code that qualifies as "clean-up operations." This includes the closing of files and other resources that were opened in the method, unlocking synchronization locks that were taken during the method, and so forth. Meanwhile, if you follow the first guideline-only catching expected exceptions-the number of methods that actually catch any exceptions at all is reduced in the end.

It's nice to see that I'm not alone thinking this way!

Monday, June 14, 2004

Try/Finally for cleaner cleanup.

One thing that people often overlook is how a try/finally block can make your code both more readable and more robust. It's a great tool for cleanup code. You can see me using it on the GetErrorMessage on my WinHTTPException class. If you look carefully, you'll see that I close the handle after the return statement.
As a sample, suppose you need to read some temporary information from a file and return it as a string. No matter what happens, you need to delete this file, because it's temporary. This kind of return & cleanup begs for a try/finally block.

Let's see the simplest possible code without using try/finally:

string ReadTempFile(string FileName)
{
string fileContents;
using (StreamReader sr = new StreamReader(FileName))
{
fileContents = sr.ReadToEnd();
}
File.Delete(FileName);
return fileContents;
}


This code also has a problem when an exception is thrown on, e.g, the ReadToEnd method. So, I've actually saw some people trying to solve it coding as this:

string ReadTempFile(string FileName)
{
try
{
string fileContents;
using (StreamReader sr = new StreamReader(FileName))
{
fileContents = sr.ReadToEnd();
}
File.Delete(FileName);
return fileContents;
}
catch (Exception)
{
File.Delete(FileName);
throw;
}
}


The code is becoming complex and it's starting to duplicate code.

Now, see how much cleaner and robust is the try/finally solution:


string ReadTempFile(string FileName)
{
try
{
using (StreamReader sr = new StreamReader(FileName))
{
return sr.ReadToEnd();
}
}
finally
{
File.Delete(FileName);
}
}


Where did the fileContents variable go? It's not necessary anymore, because we can return the contents and the cleanup code executes after the return point. This is one of the advantages of having code that can run after the function returns: you can clean resources that may be needed for the return statement.

Friday, June 11, 2004

Resharper

I just can't avoid mentioning Resharper. What a great tool!
If you don't know it yet, JetBrains is providing an Early Access Program for free.
It's a refactoring tool for C# and VS.NET, which basically makes your IDE as smart as IntelliJ.
Try it and you won't regret!

Wednesday, June 09, 2004

Case insensitive String functions

The more I use .NET, the more I become astonished about what I still don't know. No, I'm not talking about some highly-advanced features: I'm talking about simple things, as string methods.
Sometimes you may need to do a case-insensitive IndexOf, or a case-insensitive StartsWith. Most people would call ToLower or ToUpper. This has two problems:
1. This is not the same thing than a case-insensitive search on some cultures. Second
2. This is slow, and creates two more objects to be collected. If done on a loop, this can create a lot of unnecessary overhead.

The solution most people (me included) do is creating their own case-insensitive IndexOf or StartsWith.
Until I found the CompareInfo class. You can use it through CultureInfo.CurrentCulture.CompareInfo and call, e.g.:

    CultureInfo.CurrentUICulture.CompareInfo.IndexOf(text, substring, CompareOptions.IgnoreCase)

Cool, huh?
It's very fast and doesn't add overhead to the GC.

Free from the pad!

Oh, the joy!
Input and output devices should be where you spend most of your money when building a new computer: they increase productivity like anything else, and their value have a low depreciation through time.
At my office, I use two 17" monitors, a cool MS keyboard with lots of cool features. I also work on a big desk, with lots of space. But I never thought of upgrading my old, trusty Logitech mouse with a wheel: it worked, and I thought it was as good as an optical mouse, and it was only that cool red led light that attracted people. I couldn't be farther from the truth. Yesterday, I used one for a while on a coworker's machine, and when I got back to my old mouse, I couldn't like it anymore. Just the absence of the mouse pad and the freedom to move it on is worth the upgrade. Now, I bought this MS basic optical mouse: it is beautiful, cheap, and looks distinct, seems reliable and easy to install; I plugged right into one of the keyboard USB ports and it started working.

Run, my little mouse, run free from the pad!

Tuesday, June 08, 2004

Don Knuth: Musings and more

I just found a great page: Knuth: Musings and more. A series of videos featuring the great Don Knuth. I didn't watch them yet, but I plan doing it on the weekend.

Monday, June 07, 2004

Visual Studio Debugger Breakpoint Helper

Great tool!
Kant mentioned, on CP lounge, a great tool for troubleshooting breakpoints that don't work.
Visual Studio Debugger Breakpoint Helper

Win32Exceptions capable of dealing with module error messages

For those who may be, like me, in need of an exception that properly deals with Win32 error codes (obtained from GetLastError()) that can come from the system or an specific module, the code below does the trick.
You can change the name "WINHTTP" (it's what I'm dealing right now) for any DLL name you have and go for it!

using System;
using System.Text;
using System.Runtime.InteropServices;
using System.ComponentModel;

namespace WinHTTP
{
public class WinHTTPException : Win32Exception
{
[DllImport("kernel32.dll", CharSet=CharSet.Auto, SetLastError=true)]
private static extern int FormatMessage(uint dwFlags, HandleRef lpSource, uint dwMessageId, uint dwLanguageId, StringBuilder lpBuffer, uint nSize, IntPtr arguments);

[DllImport("kernel32.dll", CharSet=CharSet.Auto, SetLastError=true)]
private static extern IntPtr GetModuleHandle(string lpModuleName);

[DllImport("kernel32.dll", CharSet=CharSet.Auto)]
private static extern bool CloseHandle(HandleRef handle);

private const uint FORMAT_MESSAGE_FROM_HMODULE = 0x00000800;
private const uint FORMAT_MESSAGE_ALLOCATE_BUFFER = 0x00000100;
private const uint FORMAT_MESSAGE_IGNORE_INSERTS = 0x00000200;
private const uint FORMAT_MESSAGE_FROM_SYSTEM = 0x00001000;

public WinHTTPException()
: base(Marshal.GetLastWin32Error())
{
}

public WinHTTPException(int NativeErrorCode)
: base(NativeErrorCode)
{
}

private string GetErrorMessage()
{
IntPtr hModule = IntPtr.Zero;
try
{
hModule = GetModuleHandle("WINHTTP");
int error = NativeErrorCode;
if (hModule == IntPtr.Zero)
return "Unknown error " + error.ToString("X");
StringBuilder sb = new StringBuilder(1024);
int res = FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_FROM_HMODULE, new HandleRef(this, hModule), (uint)error, 0, sb, (uint)sb.Capacity, IntPtr.Zero);
if (res == 0)
return "Unknown error " + error.ToString("X");
else
return sb.ToString(0, res);
}
finally
{
if (hModule != IntPtr.Zero)
CloseHandle(new HandleRef(this, hModule));
}
}

public override string Message
{
get
{
return GetErrorMessage();
}
}

}
}



Saturday, June 05, 2004

Solved the Win32Exception mistery...

Great! I wrote my own exception, similar to Win32Exception, and now it's working. I'm calling FormatMessage with FORMAT_MESSAGE_FROM_HMODULE | FORMAT_MESSAGE_FROM_SYSTEM and the proper messages are being retrieved.

My managed WinHTTP wrapper starts to work...

My WinHTTP wrapper is starting to work. I even made a nicer OOP model around WinHTTP, with a Session, a Connection, a Request, a Response (WinHTTP doesn't have a Response handle, everything belongs to the Request handle), and a ResponseStream that can be used to read the page.
The exceptions aren't still being able to get the proper Win32 messages, I can't understand why, as I'm using System.ComponentModel.Win32Exception and SetLastError = True on Interop.
Now, I just need to add support to HTTP status codes, headers and I'm done.
This really begs for an article, I wish I had a bit more free time (notice, it's Saturday and I'm working...)

Friday, June 04, 2004

P/Invoke to the rescue

I just realized that I won't need to download the Platform SDK on all of our machines. I'll write a WinHTTP wrapper in C# (I sense a new CP article comming), so there'll be no need for the updated SDK on the machines.
Sure, I'll send it to PInvoke.NET

I quit!

Unfortunately, I've tried everything and it wasn't enough. The leaks may seem harmless on a desktop application, which can leak a few handles and then wait for the garbage collector to appear, but in a server application it's simply a killer. On a 4-way SMP, there may be up to 100 simultaneous request running (each request takes an average of 5 seconds to complete), and if each of them leak 30 handles, I'm leaking 600 handles/second. The GC simply isn't smart enough to clean all of them and the number never gets down.
Since Crivo only is supported on Windows 2000, XP and 2003 with the last Service Pack applied, an alternative way is by using WinHTTP. No code in Crivo calls directly the .NET HTTP libraries, we provide a smart facade (similar to System.Net.WebClient, but smarter), so it's "just" a matter of rewriting our HTTP library to use WinHTTP.
BTW, I just discovered that WinHTTP is not included on the SDK that came with VS.NET 2003, and since it's now part of the OS (a good thing, IMO), it's not available for separate download anymore. To get it now, we need to run Platform SDK update on all development machines and on the build machine (~250 MB). :sigh:
I think I'll start by writing some C++ wrapper classes. Those who worked with me in the past know that I've always been a fan of RAII.

Thursday, June 03, 2004

Damn! Handle leaks on the CLR!

On the past few days, I'm dealing with some hard to find resource leaks. We seem to be pushing some areas CLR to their limit. You may say, "hey, how can you get resource leaks in a garbage collected environment"?.
Well, the simple code below will leak lots of handles.

private void MakeARequest()
{
HttpWebRequest wr
= (HttpWebRequest)WebRequest.Create("http://www.microsoft.com/");
wr.Proxy
= WebProxy.GetDefaultProxy();
wr.Proxy.Credentials
= CredentialCache.DefaultCredentials;
using (HttpWebResponse wrp = (HttpWebResponse)wr.GetResponse())
{
using (Stream s = wrp.GetResponseStream())
{
using (StreamReader sr = new StreamReader(s))
{
string st = sr.ReadToEnd();
sr.Close();
s.Close();
}
}
}
}



Did you see it? It isn't missing a single Close() or Dispose() call, and yet leaks lots of handles. The biggest the page, the more handles it leaks.
I've been using an evaluation version (10 days remaining...) of the great SciTech's .NET memory profiler and noticed that the CLR is forgetting to dispose some (a lot!) ManualResetEvent, Sockets, and some internal Streams and Readers.
After a while, I found that the culprit are the synchronous HttpWeb(Request, Response) methods, and, by calling the assynchronous versions, I was able to reduce it a lot. Even the returned stream, System.Net.ConnectStream have these problems.
The only problem is when the request throws an Exception, it still leaks lots of handles, and, by now, it seems that I can do nothing about it...

My first post!

On this blog, I intend to start sharing some knowledge and experiences on .NET development in general.
I'd like to introduce myself, if you still don't know me from The CodeProject. If this is the case, maybe you could be interested in some of my articles.
I'm the technical architect of Crivo. Crivo is an automated credit and risk assessment system, used by several large companies in Brazil. We combine some AI techniques (perceptrons and an expert rule inference engine) with a set of drivers which can obtain information in almost any information source. This set of drivers is one of our most powerful features, and we can deliver more than 27,000 variables coming from around 30 different sources to the assessment engine so it can take a decision.
It's a huge .Net based system (around 1.3MLOC as of date), comprised of around 180 projects. And it's growing fast, as we have a very productive team. As I blog, I'll talk more about how we develop software.

This page is powered by Blogger. Isn't yours?