Smalltalk in The Cloud

WinHttpClient is a new HTTP client library for VA Smalltalk on Windows. This library uses a Windows DLL to perform the HTTP network operations. The DLL is called winhttp.dll and is available on all the current Windows platforms. Using this DLL all the features of the HTTP protocol can be used in an efficient way. This includes support for secure (https) connections.

This is a first release of the Smalltalk library; in this release not all the features are available from Smalltalk yet. But all the common functionality can already be used:

• HTTP GET, POST and HEAD requests
• Support for setting custom request headers and access to the response headers
• Support for secure http (https) including support for client certificates
• Streaming downloads and uploads
• Proxy support
• Server and proxy authentication support

We developed this library because the standard HTTP client of VA Smalltalk is not fully stable on Windows when using secure connections on multi-core systems. Nationaal Spaarfonds (a part of Delta Lloyd Group) has a VA Smalltalk application in production that needs to call secure webservices. This works fine most of the time but sometimes causes the image to freeze. See the Instantiations forum for a discussion of this problem.

WinHttpClient can be plugged into the VA Smalltalk webservice stack as a replacement for the standard HTTP client. Our stress tests show that the stability problems disappear after this switch.

The WinHttpClient library is open source and is available on VAStGoodies.com. If you are interested please try it out. If you have suggestions on how we can improve the API we would love to hear them. Contributions to add missing features are also very welcome.

WinHttpClient API

The API is structured the same way as the Windows winhttp library. You can find the documentation on MSDN: http://msdn.microsoft.com/en-us/library/aa384273(VS.85).aspx

Important concepts are Session, Connection and Request. Before you can do anything with the library you need a Session instance:

| session response |
session := WinHttpSession new.
response := session submitGetRequest: 
	'http://en.wikipedia.org/wiki/Smalltalk'

The WinHttpSession class has a number of submitXXX methods. These methods create and send HTTP requests and handle the responses. They return an instance of WinHttpResponse, this object contains the HTTP status, the HTTP header information and the contents.

If the contents of the response is possibly very large than it is better to use a streaming variant of the submit request which has a write stream as an argument. This method will still answer a WinHttpResponse instance but the contents variable of this instance will be nil.

After you have finished with the session you should send “session release”. This is because the session object contains a handle to a Windows structure that must be freed to prevent memory leaks. The class side of WinHttpSession contains a number of convenience submit methods that handle the release message for you.

WinHttpConnection

If you need to send multiple requests to the same host it is more efficient to work with a connection object instead of a session:

| session connection responses |
session := WinHttpSession new.
responses := OrderedCollection new.
connection := session connectTo: 'en.wikipedia.org'.
[
	responses 
		add: (connection submitGetRequest: '/wiki/Smalltalk');
		add: (connection submitGetRequest: '/wiki/HTTP');
		add: (connection submitGetRequest: '/wiki/Virtual_machine')
] ensure: [ connection release ].
responses

WinHttpRequest

Normally you don’t need to work with requests objects directly, you can just use the submit methods of the session or connection objects. Only when you need to set special options you need to access the request object directly. An example of this is when you want to send a request to a secure web server with an invalid server certificate. The default behavior of winhttp is to refuse the connection and throw an error. For example, if you want to accept certificates with an unknown certificate authority you need to set the “IgnoreUnknowsCA” option:

| url session connection request response |
	
url := 'https://www.securebutinvalid.com' sstAsUrl.
session := WinHttpSession new.
connection := session connectToUrl: url.
	
request := connection openGetRequest: url absolutePath.
request setSecurityOptionIgnoreUnknownCA.
request send.
response := request getResponse.

request release.
connection release.
session release.	
response

Note that we think this part of the WinHttpClient API is a bit awkward. The intention is to improve this in future versions.

Proxy support

WinHttp has support for proxy’s and a number of proxy authentication methods (Basic, NTLM etc). When you instantiate a session you can specify the proxy to use or you can use the defaultProxy class methods to set a default for all sessions.

It is also possible to copy the proxy settings that are configured in Internet Explorer. Use initProxyInfoFromIE class method of the Session class for this.

For more examples on how to use WinHttpClient see the unit tests in WinHttpClientTestApp.

Using WinHttpClient in the VA Webservice stack

The application WinHttpClientWebServiceSupportApp contains classes that can replace the standard HTTP Client with WinHttpClient for calling webservices. Thanks to the flexible way that webservice support is implemented in VA Smalltalk this is quite easy to do. We just have to replace the default HTTP and HTTPS dispatchHandlers with a custom version. We can do this when we create the webservice container:

| container httpHandler secureHttpHandler |

container := SstWSContainer containerNamed: 'test' ifNone: [ SstWSContainer createContainerNamed: 'test' ].
	
httpHandler := WinHttpDispatchHandler new.
secureHttpHandler := WinHttpDispatchHandler new.

container handlerFactory 
	register: httpHandler named: 'wsHttpClientRequestHandler' inNamespace: container handlerFactory globalNamespace;
	register: secureHttpHandler named: 'wsHttpsClientRequestHandler' inNamespace: container handlerFactory globalNamespace.
container

For a full example see the test class WinHttpWeatherForecastTest.

Secure webservices with client certificates

Supporting secure webservices that use HTTPS was very easy with WinHttpClient. But adding support for client certificates involved a bit more work. To set a client certificate for a request another Windows DLL is required: crypt32.dll. This DLL is also shipped with all the current Windows platforms. This DLL contains the functionality to handle certificates and all kinds of secure hashing, encrypting and decrypting functionality. In the future we can make all this functionality available from Smalltalk in a separate project. Currently WinHttpClient uses just a few functions from this DLL to set the correct certificate for a request.

You can tell the WinHttpDispatchHandler to use a client certificate by calling the setter method clientCertificateFilename:. The file you provide must be in one of the supported formats by Microsoft and must contain bot the certificate and the private key.

In the old situation we had the certificate and the private key in two separate pem files. WinHttp cannot handle this. We used openssl to convert the two pem files into a single pfx file:

openssl pkcs12 -export -out testcert.pfx -inkey testcertrsa.pem -certfile testcert.pem