This post will show how we can use the API’s provided by IBM i/OS to work through the objects in a library and look through the objects to find if any mismatches exist. Again we are going to use the previous Service Programs functionality to reduce the code content so make sure they are available before trying to compile this code. The code can also form the base for additional functionality such as auto compiling the programs etc. but for now we are just going to focus on finding where a signature mismatch exists. We will go through and force a mismatch using one of our existing service programs to prove the effectiveness of the check.
First we are going to need a command to front end the program, we are just going to take a single library name which is where we will search for the *PGM and *SRVPGM objects. You can expand the code to search different libraries etc but as one of our founding rules is to segregate objects such as Programs | Data | Objects | Temp. We will always have our *PGM and *SRVPGM in a single library. The point we need to make here is that a *SRVPGM may also have dependent *SRVPGM signatures so we have to check those in the same manner we check the *PGM – *SRVPGM dependencies.
IBM Provides a couple of API’s which help us to perform the functionality we need which are QBNLPGMI and QBNLSPGM use the links to check out the documentation.
This is the command we will use to front end the program.
/*===================================================================*/
/* Command name : CHKSIGS */
/* Author name..: Chris Hird */
/* Date created : March 2018 */
/* */
/* Purpose......: Check pgm and srvpgm sigs in lib */
/* CPP..........: chksigs */
/* Revision log.: */
/* Date Author Revision */
/* */
/* @Copyright Chris Hird 2018 */
/*===================================================================*/
CMD PROMPT('Check signatures')
PARM KWD(LIB) TYPE(*CHAR) LEN(10) MIN(1) CHOICE(*VALUES) PROMPT('Library to be Checked')
This is the program we will use to walk through the list of objects and verify the signatures.
// Copyright (c) 2018 Chris Hird
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions
// are met:
// 1. Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// 2. Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
//
// Disclaimer :
// This code is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#include <H/MSGFUNC> // Message headers
#include <H/SPCFUNC> // User Space functions
#include <H/COMMON> // common header file
#include <qbnlpgmi.h> // ILE pgm info
#include <qbnlspgm.h> // Srv pgm info
#include <quslobj.h> // list objects
// function pre-declarations
int dsp_sig(const char *,size_t, char *);
void Dump_Hex_Output(const char*, size_t);
int main(int argc, char **argv) {
int i = 0; // counter
int j = 0; // counter
int k = 0; // counter
short int found = 0; // found flag
char Obj_SPC[20] = "OBJLST QTEMP "; // USRSPC
char Srv_SPC[20] = "QBNLSPGM QTEMP "; // usrspc
char Pgm_SPC[20] = "QBNLPGMI QTEMP "; // usrspc
char Srv_Format1[8] = "SPGL0200"; // request format
char Srv_Format[8] = "SPGL0800"; // request format
char Pgm_Format[8] = "PGML0200"; // request format
char SPC_Desc[50] = {' '}; // USRSPC Description
char Replace[10] = "*YES "; // Replace USRSPC
char Ext_Atr[10] = "USRSPC "; // Ext attr USRSPC
char Initial_Value = ' '; // Init val USRSPC
char Auth[10] = "*CHANGE "; // Pub aut to USRSPC
char Signature[16]; // SrvPgm Signature
char Obj_Sel[20] = "*ALL "; // Objects to select
char srv_pgm_name[20]; // service program name
char sig1[33] = {0}; // signature
char sig2[33] = {0}; // signature
char msg_dta[1024]; // msg buffer
char *List_Section; // usrspc list ptr
char *tmp; // temp ptr
Qus_OBJL0100_t *Obj_Entry; // USRSPC Pointer
Qbn_LPGMI_PGML0200_t *Pgm_Entry; // info
Qbn_LSPGM_SPGL0800_t *Srv_Entry; // SrvPgm Info
Qbn_LSPGM_SPGL0200_t *SrvPgmEntry; // service program info
Qus_Generic_Header_0100_t *obj_space; // usrspc hdr ptr
Qus_Generic_Header_0100_t *srv_space; // usrspc hdr ptr
Qus_Generic_Header_0100_t *pgm_space; // usrspc hdr ptr
Os_EC_t Error_Code; // error struct
Error_Code.EC.Bytes_Provided = sizeof(Error_Code);
memcpy(&Obj_Sel[10],argv[1],10);
// build the user spaces
// holds all of the objects in the library
if(Get_Spc_Ptr(Obj_SPC,&obj_space,_16MB) != 1)
exit(-1);
// holds the Service Program details
if(Get_Spc_Ptr(Srv_SPC,&srv_space,_1MB) != 1)
exit(-1);
// holds the program details
if(Get_Spc_Ptr(Pgm_SPC,&pgm_space,_4MB) != 1)
exit(-1);
// list all objects in the library
QUSLOBJ(Obj_SPC,
"OBJL0100",
Obj_Sel,
"*ALL ",
&Error_Code);
if(Error_Code.EC.Bytes_Available > 0) {
snd_error_msg(Error_Code);
exit(-1);
}
// work through the objects
tmp = (char *)obj_space;
// move to list data
tmp += obj_space->Offset_List_Data;
// set structure pointer to list data
Obj_Entry = (Qus_OBJL0100_t *)tmp;
//printf("Number of objects listed %d\n",obj_space->Number_List_Entries);
for(i = 0; i < obj_space->Number_List_Entries; i++) {
// look only at the programs
if(memcmp(Obj_Entry->Object_Type_Used,"*PGM",4) == 0) {
// get the list of service programs
//printf("Checking *PGM %.20s\n",Obj_Entry->Object_Name_Used);
QBNLPGMI(Pgm_SPC,
Pgm_Format,
Obj_Entry->Object_Name_Used,
&Error_Code);
if(Error_Code.EC.Bytes_Available > 0) {
// send error message but do not exit!
snd_error_msg(Error_Code);
}
else {
//printf("Number of Service Programs listed %d\n",pgm_space->Number_List_Entries);
// get to the contents of the service programs
if(pgm_space->Number_List_Entries > 0) {
tmp = (char *)pgm_space;
tmp += pgm_space->Offset_List_Data;
Pgm_Entry = (Qbn_LPGMI_PGML0200_t *)tmp;
// loop through the listed service programs
for(j = 0; j < pgm_space->Number_List_Entries; j++) {
// check the signature for the service program
memcpy(srv_pgm_name,Pgm_Entry->Bound_Service_Program,10);
// check for library list as the library definition.
if(*Pgm_Entry->Bound_Service_Library_Name == 0x00)
memcpy(&srv_pgm_name[10],"*LIBL ",10);
else
memcpy(&srv_pgm_name[10],Pgm_Entry->Bound_Service_Library_Name,10);
QBNLSPGM(Srv_SPC,
Srv_Format,
srv_pgm_name,
&Error_Code);
if(Error_Code.EC.Bytes_Available > 0) {
// send error message but do not exit!
snd_error_msg(Error_Code);
}
else {
// check the signature
tmp = (char *)srv_space;
tmp += srv_space->Offset_List_Data;
Srv_Entry = (Qbn_LSPGM_SPGL0800_t *)tmp;
found = 0;
//printf("Number of signatures %d\n",srv_space->Number_List_Entries);
for(k = 0; k < srv_space->Number_List_Entries; k++) {
//printf("*PGM %.10s *SRVPGM %.20s\n",Obj_Entry->Object_Name_Used,srv_pgm_name);
//dsp_sig(Srv_Entry->Signature,16,sig1);
//dsp_sig(Pgm_Entry->Bound_Service_Signature,16,sig2);
//printf(msg_dta,"SRVPGM = %.32s\nPGM = %.32s\n",sig1,sig2);
if(memcmp(Srv_Entry->Signature,Pgm_Entry->Bound_Service_Signature,16) == 0) {
found = 1;
break;
}
Srv_Entry++;
}
if(found == 0) {
// signatures do not match so print to stdout
printf("No valid signature in %.10s for Pgm %.10s\n",srv_pgm_name,Obj_Entry->Object_Name_Used);
printf("*PGM signature - ");
Dump_Hex_Output(Pgm_Entry->Bound_Service_Signature,16);
printf("\n");
}
}
Pgm_Entry++;
}
}
}
}
// if its a service program
else if(memcmp(Obj_Entry->Object_Type_Used,"*SRVPGM",7) == 0) {
//printf("Checking *SRVPGM %.20s\n",Obj_Entry->Object_Name_Used);
QBNLSPGM(Pgm_SPC,
Srv_Format1,
Obj_Entry->Object_Name_Used,
&Error_Code);
if(Error_Code.EC.Bytes_Available > 0) {
// send error message but do not exit!
snd_error_msg(Error_Code);
}
else {
//printf("Number of Service Programs listed %d\n",pgm_space->Number_List_Entries);
if(pgm_space->Number_List_Entries > 0) {
tmp = (char *)pgm_space;
tmp += pgm_space->Offset_List_Data;
SrvPgmEntry = (Qbn_LSPGM_SPGL0200_t *)tmp;
for(j = 0; j < pgm_space->Number_List_Entries; j++) {
// check the signature for the service program
memcpy(srv_pgm_name,SrvPgmEntry->Bound_Srvpgm_Name,10);
if(*SrvPgmEntry->Bound_Srvpgm_Library_Name == 0x00)
memcpy(&srv_pgm_name[10],"*LIBL ",10);
else
memcpy(&srv_pgm_name[10],SrvPgmEntry->Bound_Srvpgm_Library_Name,10);
QBNLSPGM(Srv_SPC,
Srv_Format,
srv_pgm_name,
&Error_Code);
if(Error_Code.EC.Bytes_Available > 0) {
// send error message but do not exit!
snd_error_msg(Error_Code);
}
else {
// check the signature
tmp = (char *)srv_space;
tmp += srv_space->Offset_List_Data;
Srv_Entry = (Qbn_LSPGM_SPGL0800_t *)tmp;
found = 0;
//printf("Number of signatures %d\n",srv_space->Number_List_Entries);
for(k = 0; k < srv_space->Number_List_Entries; k++) {
//printf("*PGM %.10s *SRVPGM %.20s\n",Obj_Entry->Object_Name_Used,srv_pgm_name);
//dsp_sig(Srv_Entry->Signature,16,sig1);
//dsp_sig(Pgm_Entry->Bound_Service_Signature,16,sig2);
//printf("SRVPGM = %.32s\nPGM = %.32s\n",sig1,sig2);
if(memcmp(Srv_Entry->Signature,SrvPgmEntry->Bound_Srvpgm_Signature,16) == 0) {
found = 1;
break;
}
Srv_Entry++;
}
if(found == 0) {
printf("No valid signature in %.10s for Pgm %.10s\n",srv_pgm_name,Obj_Entry->Object_Name_Used);
printf("\n*PGM signature - ");
Dump_Hex_Output(SrvPgmEntry->Bound_Srvpgm_Signature,16);
printf("\n");
}
}
SrvPgmEntry++;
}
}
}
}
Obj_Entry++;
}
exit(0);
}
// functions dsp_sig()
// Purpose: Create Hex signature for display
// @parms
// Input buffer (signature)
// length of Buffer
// sig buffer (written from input content)
// returns int
int dsp_sig(const char *buf,size_t buf_len, char *sig) {
int i = 0; // counter
while(buf_len > 0) {
sprintf(&sig[i],"%02x",*buf);
i+= 2;
buf_len--;
buf++;
}
return 1;
}
// functions Dump_Hex_Output()
// Purpose: prints the Hex content to stdout
// @parms
// Input Buffer
// Length of input buffer
// returns void
void Dump_Hex_Output(const char* buf, size_t buf_len) {
while(buf_len > 0) {
printf("%02x",*buf);
buf_len--;
buf++;
}
}
OK so lets compile the above as before so we can test against our existing library.CRTCMD CMD(OSPGM/CHKSIGS) PGM(*LIBL/CHKSIGS) SRCFILE(OSLIB/QCMDSRC) SRCMBR(CHKSIGS) REPLACE(*YES)If you prompt the command and run against the OSPGM Library you will see nothing happens which is what you would expect at this stage as we have recently updated the service programs and we know that all is well. But before we change things to force the error to occur let’s review the code and see exactly what is going on in the code.
CRTCMOD MODULE(OSLIB/CHKSIGS) SRCFILE(OSLIB/QCSRC) SRCMBR(CHKSIGS) DBGVIEW(*SOURCE) REPLACE(*YES) TGTRLS(V7R1M0)
CRTPGM PGM(OSPGM/CHKSIGS) MODULE(OSLIB/CHKSIGS) BNDDIR(OSLIB/OS) TGTRLS(V7R1M0)
The first thing we did different this time is added a couple of new functions which are part of the program, we have added them after they are called (dsp_sig() and Dump_Hex_Output()). This would cause an error in the compiler as we create the mod because we are trying use the functions before the compiler knows where they are. To rectify that, just before the main() entry point we have added a couple of pre-declarations for the functions. This tells the compiler exactly what those functions are expecting as parameters in much the same way we do that with the Service Program Functions when we include the header file. The binder will find the functions as well as they are contained in the same code listing as they are used.
Hopefully you have reviewed the API information using the links above, you will notice that we used 3 different formats to call the APIs, 1 format for QNBLPGMI and 2 formats for QNBLSPGM. The ‘200’ formats pull the same kind of information but for different object type (*PGM / *SRVPGM) but the ‘800’ returns a list of the signatures stored in the *SRVPGM object for its exports (no such thing in a program ;-)). So with that information in hand we can dump a list of all of the objects in the library using the same QUSLOBJ API we have used previously.
We will walk through the list of objects in a ‘for’ loop because the API tells us exactly how many objects are in the list (The special structure and pointers for list APIs is clearly defined in the documentation so if you want to know more feel free to go through the documentation). The header gives us a number of offsets we can use to line up the relevant pointers which will allow us to walk through the object information. We chose a format “OBJL0100” which returns the least amount of information about each object which makes things faster plus we only need the Object Name and Object Type for this program.
NOTE: The pointers for these API’s have fully defined structures so we can simply use the struct_ptr++ notation to tell it is move the start of the structure pointer to the next start point by adding the size of the structure. Some structures in the header are incomplete because they have some data which is not known at the time of compiling the program so using the struct_ptr++ notation will possibly result in incorrect data being returned.
Then we use the information to determine if the object is a *PGM or a *SRVPGM, we need this information to allow us to call the correct API and format. You will see for a *PGM object we first call the QBNLPGMI API, this provides a list of Service Programs that are bound to it, then we get the informaton about the *SRVPGM listed using the QBNLSPGM API. If its a *SRVPGM we have to first call the QBNLSPGM API with a format of ‘SPGL0200’ then call the API again with the ‘SPGL0800’ for each of the Service Programs which are bound to the listed service Program.
You will see we have a number of nested loops in the program, first is a loop through the list of objects (we are using ‘i’ as the counter) then we have the list of Service Programs bound to the *PGM or *SRVPGM object (we are using ‘j’ as the counter) and finally we have the list of signatures in the actual *SRVPGM to walk through (we are using ‘k’ as the counter). I have made the mistake of using the same counter in a deeper listed loop in the past, as you can imagine it can create some strange results.
The 2 inner loops are doing a similar process except the ‘j’ loop is working over the list of *SRVPGMs from the output of 2 separate APIs (QBNLPGMI or QBNLSPGM). The ‘k’ loop is always over the signatures for the Service program which is returned from the QBNLSPGM API. We have had to add a flag to the Signature check as we need to know if a valid signature is in the Service Program that matches to the signature found in the *SRVPGM/*PGM object we are checking. If we find a match we automatically drop out of the for loop as there is not much point in checking any other signatures that are logged in the Service Program.
The next question is how do we verify that the program is actually doing exactly what it is intended to do (find *PGM/*SRVPGMs which have a signature for a particular *SRVPGM that is not contained in that *SRVPGM).
You will notice that we have a number of statements which have been commented out for printf calls, you could simply un-comment out those lines are analyse the output which is generated. That would show how the program is functioning plus show where each signature is verified. Another option would be to put the program into debug and walk through the code step by step.
We are going to simply force an error which will be flagged by the program. We will do this by re-compiling the MSGFUNC Service program so it only has one Signature within it, this will then force an error because we have a number of objects which were created and bound to the Service Program while the new signature was available. The use of the QSRVSRC file is how we managed the addition of a new signature while still keeping old signature valid (older *PGM/*SRVPGMs were bound using the older signature). All we need to do in this instance is compile the Service Program using an updated QSRVSRC File.
Before we do that we are going to rename the existing *SRVPGM Object MSGFUNC, this will allow us to rename it back again to retest without having to re-comple it all over again against the QSRCSRC file. (we have to delete the old *SRVPGM to compile the new one so renaming it will help speed things up.). We will also change the QSRVSRC File Member for MSGFUNC to export a single function (this is how was in the very beginning) to reflect the following content.
STRPGMEXP PGMLVL(*CURRENT)
/********************************************************************/
/* *MODULE MSGFUNC OSLIB 03/13/18 12:02:21 */
/********************************************************************/
EXPORT SYMBOL("snd_error_msg")
ENDPGMEXP
RNMOBJ OBJ(OSPGM/MSGFUNC) OBJTYPE(*SRVPGM) NEWOBJ(MSGFUNC_O)If you display the signatures for the Service Program
CRTSRVPGM SRVPGM(OSPGM/MSGFUNC) MODULE(OSLIB/MSGFUNC) SRCFILE(OSLIB/QSRVSRC) BNDDIR(OSLIB/OS) TGTRLS(V7R1M0)
DSPSRVPGM SRVPGM(OSPGM/MSGFUNC) DETAIL(*SIGNATURE)You will notice that the *SRVPGM only has a single signature
Display Service Program Information
Display 1 of 1
Service program . . . . . . . . . . . . : MSGFUNC
Library . . . . . . . . . . . . . . . : OSPGM
Owner . . . . . . . . . . . . . . . . . : CHRISH
Service program attribute . . . . . . . : CLE
Detail . . . . . . . . . . . . . . . . . : *SIGNATURE
Signatures:
00000087A2946D99969999856D8495A2
If we look at the old function you will notice that the signature is the same as one of the signatures. The signature matches the *PREVIOUS signature of the later Service Program.
DSPSRVPGM SRVPGM(OSPGM/MSGFUNC_O) DETAIL(*SIGNATURE)
Display Service Program Information
Display 1 of 1
Service program . . . . . . . . . . . . : MSGFUNC_O
Library . . . . . . . . . . . . . . . : OSPGM
Owner . . . . . . . . . . . . . . . . . : CHRISH
Service program attribute . . . . . . . : CLE
Detail . . . . . . . . . . . . . . . . . : *SIGNATURE
Signatures:
00000087A2946D999E13C2CB45CDEFC2
00000087A2946D99969999856D8495A2
Now when we run our test against the OSPGM library we will get output similar to the following.
No valid signature in MSGFUNC for Pgm LSTOBJBYTP *PGM signature - 00000087a2946d999e13c2cb45cdefc2 Press ENTER to end terminal session.This is because we created the LSTOBJBYTP program using the later copy of the Service Program (one with 2 signatures and the latest one matched). You can see the signature matches the one which was created with the new exports (see previous post), so we now know the program works as we intended.
Lets set everything back to the way it was.
DLTOBJ OBJ(OSPGM/MSGFUNC) OBJTYPE(*SRVPGM)If you updated the QSRVSRC File update back again as we will use it later in the series.
RNMOBJ OBJ(OSPGM/MSGFUNC_O) OBJTYPE(*SRVPGM) NEWOBJ(MSGFUNC)
Now run the test again and you will see that everything runs without any errors being returned.
Before we leave the subject we should point out the problem where you create the *SRVPGM object using the command line and ignore the QSRVSRC File. You would think that the signature which is created would match one of the ones which was created using the QSRVSRC File? but it does not, if you create the *SRVPGM using the command option to export *ALL you end up with a different signature. You would have thought that using the option *ALL in the command would result in the same signature as the one created when the QSRVSRC file has the same exports listed ie:
STRPGMEXP PGMLVL(*CURRENT)
/********************************************************************/
/* *MODULE MSGFUNC OSLIB 03/26/18 14:38:31 */
/********************************************************************/
EXPORT SYMBOL("snd_msg")
EXPORT SYMBOL("snd_error_msg")
ENDPGMEXP
But is does not, it has a VERY similar signature but not the same.0000087a2946d99969203aea45cdefc2So always ensure you use the QSRVSRC File to manage your Service Program signatures and it should help with the signature management.
This was quite a big post again! I think the *SRVPGM object and its use in application development is a major benefit for the developer and the IBM i.
Before we moved our HA4i product to a *SRVPGM model the size of the packaged product was around 120MB packed into the save file, after it reduced to just 7MB, that’s a huge saving in storage plus the memory footprint also decreased when the application was running… All of this adds up to a leaner better performing product. Another plus for us is the reduced support and management of the application that we have to do as we have a much smaller code base and less chances of stale code causing errors in different programs.
In this post we have looked at the use of 2 new API’s, nested loops in ‘C’, internal functions and how to review the signatures stored in the *SRVPGM/*PGM objects. We have also seen how to use the ‘standard’ headers which are implemented by IBM for User Space content returned using their APIs (remember not all IBM APIs return the same format when using User Spaces). I hope it has cleared some things up, I know I always learn a lot more by simply doing posts like this because it forces me to re-confirms what I think I know.
If you have any questions or improvements you think should be added let me know. I will update the code on GitHub so you have the complete code set so far. Maybe as a challenge you could implement a process to go through a number of libraries or even check *SRVPGM objects which are contained in a totally different Library to the *PGM (Hint: when you get the service program listing from the *PGM object it shows the library where the Service Program was at time of binding). As you can see a few simple rules makes things like this a much easier task :-).
Happy Easter to those who celebrate it and please let me know if I am on the right track and the content is relevant.
Chris…