Work with the Headless Web

Article ID: 56931

Antonio Mira, who wrote a question that I described in the June issue of My - i .NET, follows up with another query: "A colleague of mine generates a file with a list of URLs on the AS/400. He then downloads the list of URLs to an Excel spreadsheet. Each URL has a list of parameters embedded in it. If I open the Excel spreadsheet and click on one of the URLs, Internet Explorer is invoked and that triggers a phone call to a number, which is embedded in the URL. As you can see, this process is manual. Is there a way to write a VB program or in any other platform that will 'hit' each address without having to invoke Internet Explorer?"

From what I gather in Antonio's message, the calls from the URLs are used to invoke programs on another server. Most likely, the required function is to simply place the phone call without producing a response HTML page. This sounds very much like a web service, and I suppose if I were tasked with providing a telephone calling function accessible to my applications via the web, I would look first for a web service. In fact, what caught my interest about this question is that I have used a web service that does outbound telephone dialing from cdyne.com (go to their site at www.cdyne.com and look for the "Phone Notify" web service).

However, Antonio does not indicate that he is accessing a web service, simply that he has an invoked URL that passes parameters. It turns out that .NET provides classes you can use to programmatically call upon a URL. You can also receive the response into your program. Essentially, you can write code that replaces the functions normally performed by a browser. After you see how to send a URL request and work with an HTTP response, you can write programs to process lists of URLs, similar to Antonio's requirement.

Before getting too much further along, you should note that there is nothing unique to .NET in this capability. Indeed, Scott Klement provides the HTTPAPI library to provide similar functionality on the IBM i. So if you need to directly invoke URLs and work with HTTP responses from IBM i programs, you should start by looking into HTTPAPI.

An Example Program

In order to have some tie-in to the IBM i, I tested a Visual Basic program that invokes an RPG-CGI program running on my IBM i. The program has previously been used for an AJAX application; as such, it does not generate an HTML page as a response but simply returns a stream of data, with each field in the stream delimited with the vertical bar (|) character. The RPG-CGI program was written with the CGIDEV2 library, which is freely available from IBM. I don't go into details about using CGIDEV2 in this article; you can find much more information about CGIDEV2 on the System iNetwork website and from other sources.

You can see the RPG-CGI program (CUSTMAST) in Figure 1. This is a simple program that receives a parameter from a URL request. The parameter is named "cust," and it contains a customer number. It uses the customer number to chain to the QCUSTCDT file. It then formats the data for the record into the delimited text string and returns that string to the requester. As you can see, the actual program code is quite simple although it does require that you have some familiarity with the CGIDEV2 techniques.

One of the advantages of CGIDEV2 for RPG web programming is that it makes it easy to generate web responses. You can create a template file with substitution variables; after plugging in your substitution values, CGIDEV2 returns the resolved template file as your web response. I show the template file that I used for this application in Figure 2. It's pretty easy to figure out how this template file works.

Figure 3 shows an example of the Console output that is produced when I run my Visual Basic program. Because I am making an HTTP request, I get HTTP response headers and a response status. The response headers, although interesting, are not important in this application.

The response status is more important. You should have code in your program that looks for the "OK" response. Any other response probably means that you don’t have any response data to work with.

The response data shows the unparsed string of delimited data that is returned as the output of the RPG program. The parsed data shows that I can break the response string back into its constituent fields so that the values are now available to my .NET application.

The Visual Basic program in Figure 4 is copied almost verbatim from the MSDN documentation for the System.Net.WebRequest class. If you need a C# example, you can find it in the documentation for that class.

As shown in the Visual Basic program, you can invoke a URL by using the WebRequest.Create method followed by an invocation of the WebRequest.GetResponse() method. If you are not expecting a response, you're done at that point: the request has been invoked via the URL.

For my example, I want to work with the response, so I access the HttpWebResponse.Headers collection and then the StatusCode and StatusDescription. To get the response data, I use the HttpWebResponse.GetResponseStream() method. After I've read the response into a string variable in my program, I use the String.Split method to split the response on the vertical bar character.

Where Can You Use This?

To be honest, I don’t know where I would recommend this technique. It is a valid method, but I have not needed such a procedure to date.

However, you might consider using this as a lightweight alternative to web services for in-house work. By that I mean that this technique does not conform in any way to standard web services; for example, there is no WSDL document that can be used to generate web service client programs.

However, look at what I was able to do with just a few lines of code: I basically have a call/parm mechanism from .NET up to the IBM i with the ability to send any number of parameters and get back a response in any format that I want. I don't need to use the .NET data provider or the OLE DB provider to call the program as a stored procedure. I'm not using any iSeries Access APIs to call programs. Everything is done in terms of HTTP request and response processing.

What's really going on is that I have TCP/IP sockets programs without having to develop complicated sockets programming in Visual Basic or in RPG. The actual sockets calls are delegated to the .NET Framework classes (on the PC) and to the IBM i Apache web server. Rather than have to get involved with low-level techniques to call a program, I simply use the code in terms of a request/response mechanism. If you know RPG, you'll have to agree that the RPG program that I'm calling is very lightweight.

Perhaps this will help Antonio and his fellow developers look at some alternative techniques. If you're able to install CGIDEV2 on your IBM i and get a web page produced from an RPG-CGI program, you can try the techniques shown in this article.

ProVIP Sponsors

ProVIP Sponsors