Date: Thu, 28 Mar 2024 17:05:30 +0000 (UTC) Message-ID: <691174113.6682.1711645530060@ip-10-10-7-29.ec2.internal> Subject: Exported From Confluence MIME-Version: 1.0 Content-Type: multipart/related; boundary="----=_Part_6681_176508358.1711645530057" ------=_Part_6681_176508358.1711645530057 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Location: file:///C:/exported.html
In grouper 2.3.0 (unpatched) and previous versions, the subject API was = configured with a sources.xml . This needs to be converted to config = overlays in 2.4.0 so Grouper packaging can have defaults and overlays. &nbs= p;Also so the Grouper team can manage internal sources without having to me= rge with a potentially changed file.
Run the grouperInstaller, admin task, upgradeTasks, convert sources xml = to properties
Follow the prompt to identify the location of the sources.xml file
This will generate the subject.properties file based on your sources.xml= edits
[appadm= in@i2midev1 patchesAuto]$ java -jar grouperInstaller.jar Do you want to 'install' a new installation of grouper, 'upgrade' an existi= ng installation, 'patch' an existing installation, 'admin' utilities, or 'createPatch' for= Grouper developers (enter: 'install', 'upgrade', 'patch', 'admin', 'createPatch' or blank fo= r the default) [install]: admin What admin action do you want to do (manage, upgradeTask)? : upgradeTask What upgrade task do you want to do (convertEhcacheXmlToProperties, convert= SourcesXmlToProperties)? : convertSourcesXmlToProperties Note, you need to convert the sources.xml file for each Grouper runtime, e.= g. loader, WS, UI. Note, to use subject sources from subject.properties, you need to be runnin= g Grouper 2.3.0+ with API patch 40 installed. Enter the location of the sources.xml file: /opt/tomcats/tomcat_f/webapps/g= rouper_v2_3/WEB-INF/classes/sources.xml Enter the location of the subject.base.properties file [/opt/tomcats/tomcat= _f/webapps/grouper_v2_3/WEB-INF/classes/subject.properties]:=20 File was written: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classe= s/subject.properties You should archive your sources.xml and remove it from your project since i= t is now unused: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/sources.xml
Sample sources.xml config
<?xm= l version=3D"1.0" encoding=3D"utf-8"?> <sources> <source adapterClass=3D"edu.internet2.middleware.grouper.subj.GrouperJd= bcSourceAdapter"> <id>jdbc</id> <name>Example JDBC Source Adapter</name> <type>person</type> =20 <init-param> <param-name>sortAttribute1</param-name> <param-value>LOGINID</param-value> </init-param> <init-param> <param-name>searchAttribute0</param-name> <param-value>searchAttribute0</param-value> </init-param> <internal-attribute>searchAttribute0</internal-attribute> <search> <searchType>searchSubject</searchType> <param> <param-name>sql</param-name> <param-value> select s.subjectid as id, s.name as name, (select sa2.value from subjectattribute sa2 where name=3D'name' and sa2.= SUBJECTID =3D s.subjectid) as lfname, (select sa3.value from subjectattribute sa3 where name=3D'loginid' and s= a3.SUBJECTID =3D s.subjectid) as loginid, (select sa4.value from subjectattribute sa4 where name=3D'description' a= nd sa4.SUBJECTID =3D s.subjectid) as description, (select sa5.value from subjectattribute sa5 where name=3D'email' and sa5= .SUBJECTID =3D s.subjectid) as email from subject s where {inclause} </param-value> </param> <param> <param-name>inclause</param-name> <param-value> s.subjectid =3D ? </param-value> </param> </search> </source> </sources>
Sample subject.properties config:
#######= ################################## ## Configuration for source id: jdbc ## Source configName: jdbc ######################################### subjectApi.source.jdbc.id =3D jdbc # this is a friendly name for the source subjectApi.source.jdbc.name =3D Example JDBC Source Adapter # type is not used all that much. Can have multiple types, comma separate.= Can be person, group, application subjectApi.source.jdbc.types =3D person # the adapter class implements the interface: edu.internet2.middleware.subj= ect.Source # adapter class must extend: edu.internet2.middleware.subject.provider.Base= SourceAdapter # edu.internet2.middleware.grouper.subj.GrouperJdbcSourceAdapter2 : if do= ing JDBC this should be used if possible. All subject data in one table/vi= ew. # edu.internet2.middleware.grouper.subj.GrouperJdbcSourceAdapter : oldes= t JDBC source. Put freeform queries in here # edu.internet2.middleware.grouper.subj.GrouperJndiSourceAdapter : used = for LDAP subjectApi.source.jdbc.adapterClass =3D edu.internet2.middleware.grouper.su= bj.GrouperJdbcSourceAdapter subjectApi.source.jdbc.param.jdbcConnectionProvider.value =3D edu.internet2= .middleware.grouper.subj.GrouperJdbcConnectionProvider subjectApi.source.jdbc.param.identifierAttributes.value =3D LOGINID # subject identifier to store in grouper's member table. this is used to i= ncrease speed of loader and perhaps for provisioning # you can have up to max 1 subject identifier subjectApi.source.jdbc.param.subjectIdentifierAttribute0.value =3D LOGINID #searchSubject: find a subject by ID. ID is generally an opaque and perman= ent identifier, e.g. 12345678. # Each subject has one and only on ID. Returns one result when searching = for one ID. # sql is the sql to search for the subject by id should use an {inclause} subjectApi.source.jdbc.search.searchSubject.param.sql.value =3D select s= .subjectid as id, s.name as name, (select sa2.value from subjectattribut= e sa2 where name=3D'name' and sa2.SUBJECTID =3D s.subjectid) as lfname, = (select sa3.value from subjectattribute sa3 where name=3D'loginid' and sa3.= SUBJECTID =3D s.subjectid) as loginid, (select sa4.value from subjectatt= ribute sa4 where name=3D'description' and sa4.SUBJECTID =3D s.subjectid) as= description, (select sa5.value from subjectattribute sa5 where name=3D'= email' and sa5.SUBJECTID =3D s.subjectid) as email from subject s where = {inclause} # inclause allows searching by subject for multiple ids or identifiers in o= ne query, must have {inclause} in the sql query, # this will be subsituted to in clause with the following. Should use a= question mark ? for bind variable subjectApi.source.jdbc.search.searchSubject.param.inclause.value =3D s.subj= ectid =3D ? # internal attributes are used by grouper only not exposed to code that use= s subjects. comma separated subjectApi.source.jdbc.internalAttributes =3D searchAttribute0
Note, the part that links the properties entries together is the part be= tween "source" and the following dot. In this case "jdbc". This= was automatically converted from the sources.xml file by looking at the so= urce name and making sure there are no special chars (replace with undersco= re).
If you want to use an xml configuration (generally you wont need to do t= his) set this in subject.properties (or remove to use the default):
#######= ################################# ## Use old sources.xml ######################################## # enter the location of the sources.xml. Must start with classpath: or fil= e: # blank means dont use sources.xml, use subject.properties # default is: classpath:sources.xml # e.g. file:/dir1/dir2/sources.xml subject.sources.xml.location =3D classpath:sources.xml
If you upgrade to 2.3.1+ it will walk you through the process of automat= ically converting your sources.xml to subject.properties
[appadm= in@i2midev1 patchesAuto]$ rm grouperInstaller.jar [appadmin@i2midev1 patchesAuto]$ wget --no-check-certificate https://softwa= re.internet2.edu/grouper/release/2.3.0/grouperInstaller.jar
Backup your sources.xml and sources.example.xml file
[appadm= in@i2midev1 patchesAuto]$ cp /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB= -INF/classes/sources.xml /tmp [appadmin@i2midev1 patchesAuto]$ cp /opt/tomcats/tomcat_f/webapps/grouper_v= 2_3/WEB-INF/classes/sources.example.xml /tmp
Install the 2.3.0#40 API patch (note: Force install that patch sinc= e it will say file mismatch)
Patch g= rouper_v2_3_0_api_patch_40 is medium risk, is not a security patch GRP-1454: migrate from sources.xml to hierarchical properties configuration - set property: grouper_v2_3_0_api_patch_40.date from: 2017/01/07 22:43:45= to: 2017/01/07 22:44:38 This patch requires all processes that user Grouper to be stopped. Please stop these processes if they are running and press <enter> t= o continue... <using default which is blank due to grouperInstaller.autorun.useDefault= sAsMuchAsAvailable and grouperInstaller.autorun.continueAfterPatchStopProce= sses>:=20 Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/subject/provider/SourceManager$SourceManagerStatusB= ean.class Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/subject/provider/NullSourceAdapter.class Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/subject/provider/NullSourceAdapter.java Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/subject/provider/LdapSourceAdapter.class Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/subject/provider/SourceManager.class Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/subject/provider/JDBCSourceAdapter.class Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/subject/provider/SourceManager.java Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/subject/provider/JNDISourceAdapterLegacy.java Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/subject/provider/JNDISourceAdapterLegacy.class Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/subject/provider/JDBCSourceAdapter.java Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/subject/provider/LdapSourceAdapter.java Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/subject/config/SubjectConfig.java Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/subject/config/SubjectConfig.class Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/grouper/GrouperSourceAdapter$2.class Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/grouper/subj/InternalSourceAdapter.java Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/grouper/subj/InternalSourceAdapter.class Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/grouper/misc/GrouperCheckConfig.class Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/grouper/misc/GrouperCheckConfig$1.class Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/grouper/misc/GrouperCheckConfig.java Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/grouper/GrouperSourceAdapter.class Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/grouper/GrouperSourceAdapter$3.class Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/grouper/GrouperSourceAdapter$1.class Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/e= du/internet2/middleware/grouper/GrouperSourceAdapter.java Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/s= ubject.base.properties Applying file: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/lib/commo= ns-digester.jar Patch successfully applied: grouper_v2_3_0_api_patch_40 - set property: grouper_v2_3_0_api_patch_40.state from: error to: applied<= /pre>
At this point, you can use Grouper as you did before, with the sources.x= ml. but if you want to use the subject.properties, you can continue. = You can see its reading from sources.xml in the startup output
subject= .properties read from: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/c= lasses/subject.properties sources.xml read from: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WE= B-INF/classes/sources.xml sources configured in: sources.xml sources.xml groupersource id: g:gsa sources.xml groupersource id: grouperEntities sources.xml jdbc source id: jdbc: GrouperJdbcConnectionProvider
Run the grouperInstaller, admin task, upgradeTasks, convert sources xml = to properties
Follow the prompt to identify the location of the sources.xml file
This will generate the subject.properties file based on your sources.xml= edits
[appadm= in@i2midev1 patchesAuto]$ java -jar grouperInstaller.jar Do you want to 'install' a new installation of grouper, 'upgrade' an existi= ng installation, 'patch' an existing installation, 'admin' utilities, or 'createPatch' for= Grouper developers (enter: 'install', 'upgrade', 'patch', 'admin', 'createPatch' or blank fo= r the default) [install]: admin What admin action do you want to do (manage, upgradeTask)? : upgradeTask What upgrade task do you want to do (convertEhcacheXmlToProperties, convert= SourcesXmlToProperties)? : convertSourcesXmlToProperties Note, you need to convert the sources.xml file for each Grouper runtime, e.= g. loader, WS, UI. Note, to use subject sources from subject.properties, you need to be runnin= g Grouper 2.3.0+ with API patch 40 installed. Enter the location of the sources.xml file: /opt/tomcats/tomcat_f/webapps/g= rouper_v2_3/WEB-INF/classes/sources.xml Enter the location of the subject.base.properties file [/opt/tomcats/tomcat= _f/webapps/grouper_v2_3/WEB-INF/classes/subject.properties]:=20 File was written: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classe= s/subject.properties You should archive your sources.xml and remove it from your project since i= t is now unused: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/classes/sources.xml
Examine the subject.properties file. Diff your sources.xml file wi= th your sources.example.xml (if exists or get here) file and make sure those diffs are expec= ted and listed in your subject.properties file
[appadm= in@i2midev1 patchesAuto]$ more /opt/tomcats/tomcat_f/webapps/grouper_v2_3/W= EB-INF/classes/properties.properties [appadmin@i2midev1 patchesAuto]$ wget --no-check-certificate https://github= .com/Internet2/grouper/raw/GROUPER_2_3_BRANCH/grouper/conf/sources.example.= xml [appadmin@i2midev1 patchesAuto]$ diff sources.example.xml /opt/tomcats/tomc= at_f/webapps/grouper_v2_3/WEB-INF/classes/sources.xml
Delete the sources.xml and sources.example.xml (if exists) files, archiv= ing if you like
[appadm= in@i2midev1 patchesAuto]$ mv /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB= -INF/classes/sources.xml ~/sources.xml.20170107.grouper2_3 [appadmin@i2midev1 patchesAuto]$ rm /opt/tomcats/tomcat_f/webapps/grouper_v= 2_3/WEB-INF/classes/sources.example.xml rm: cannot remove `/opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/class= es/sources.example.xml': No such file or directory [appadmin@i2midev1 patchesAuto]$
Install the patch in your other envs (UI, WS, daemon, etc), and copy the= subject.properties to each env (no need to go through this same process un= less you have different cache customizations in each env. Delete the = sources.xml and sources.example.xml in other envs too
Grouper startup will show the subject.properties info
subject= .properties read from: /opt/tomcats/tomcat_f/webapps/grouper_v2_3/WEB-INF/c= lasses/subject.properties sources configured in: subject.properties subject.properties groupersource id: g:gsa subject.properties groupersource id: grouperEntities subject.properties jdbc source id: jdbc: GrouperJdbcConnectionProvider