Child pages
  • Grouper internationalization
Skip to end of metadata
Go to start of metadata

This page presents information on Grouper and internationalization, handling non-English characters, UTF-8, accents, etc.

Grouper and non-English characters

Grouper works with non-English characters, but the environment must be setup correctly for this to happen.  Your database must support it, your Java settings must be set correctly, your Tomcat settings must be set correctly.  In Grouper 2.2.1 patch grouper_v2_2_1_api_patch_9 there are checks at startup. Grouper patches are listed on the Release Notes page.

There are checks at startup to make sure the non-Tomcat settings are correct.

Also, the default for grouper.base.properties for file encoding was changed. If you don't need non-English chars and you are worried about the risk of this change, please change it in grouper.properties

# when reading writing files from util classes, this is encoding (was ISO-8859-1)
grouper.default.fileEncoding = UTF-8

 

These settings will detect if your database and file system can handle UTF-8, and if transactions, and case sensitivity are handled correctly. You can get a success message if you are debugging, but this defaults to off. These will run in another thread so they don't affect start time (optionally default to true)

# if should check database and utf in new thread
configuration.checkDatabaseAndUtf.inNewThread = true
# if grouper should check to see if the database has case sensitive selects
configuration.detect.db.caseSensitive.problems = true
configuration.display.db.caseSensitive.success.message = false
# if grouper should check to see if utf-8 works on startup in files
configuration.detect.utf8.file.problems = true
# if grouper should check to see if utf-8 works on startup in the database
configuration.detect.utf8.problems = true
configuration.display.utf8.success.message = false
# if grouper in the utf8 check will check to see if grouper supports transaction
configuration.detect.db.transaction.problems = true
configuration.display.transaction.success.message = false

 

If there is a problem with start, you will see a message like this, but it will not cause grouper not to start:

Error: Cannot properly read UTF string from database: '???????', make sure your database has UTF tables and perhaps a hibernate.connection.url in grouper.hibernate.properties

Note, it will try to give suggestions about what the problem could be.

Grouper and UTF-8

utf8mb4 is not recommended because the database engines in MySQL/MariaDB that support transactions have limitations on the length of prefixes for indices. For InnoDB and XtraDB, the prefix is limited to 767 bytes. When using utf8mb4, where 4 bytes are used instead of 2, the prefixes used for some Grouper tables are too long.

 

For Grouper to handle UTF-8 characters:

  • Make sure java starts with this setting: -Dfile.encoding=UTF-8   for example, in tomcat, put something like this in your startup script:

    export JAVA_OPTS="-server -Xms50M -Xmx200M -XX:MaxPermSize=95M -Dfile.encoding=UTF-8"
  • Make sure URIEncoding is set to UTF-8 in the server.xml in tomcat in the Connector element, here is an example

    <Connector port="8111" protocol="AJP/1.3" request.tomcatAuthentication="false"
            URIEncoding="UTF-8" />
  • Make sure your database tables are using UTF-8.  For Mysql you might want to use utf8mb4. If not, e.g. for MySQL, you might need to SQL dump, delete the DB, create with default UTF-8 bin collation, and import the SQL.
  • Make sure your connect string to the database is configured to use UTF-8 (if applicable).  For MySQL, here is an example (see the stuff after the question mark).  Note, in Windows MySQL this isn't necessary

    hibernate.connection.url = jdbc:mysql://localhost:3306/grouper_v2_2?CharSet=utf8&useUnicode=true&characterEncoding=utf8
    
    

 

Grouper v2.2+ translations

If you want to add a localization to the Grouper UI text (example for French):

Add this to your grouper-ui.properties (index must increment from the base file:

# language for this bundle
grouper.text.bundle.1.language = fr

# country for this bundle
grouper.text.bundle.1.country = fr

# filename in the package grouperText that is before the .base.properties, and .properties
grouper.text.bundle.1.fileNamePrefix = grouperText/grouper.text.fr.fr

Add a grouperText/grouper.text.fr.fr.base.properties and grouper.text.fr.fr.properties

Have this at the top of grouper.text.fr.fr.base.properties

########################################
## Config chaining hierarchy
########################################

# comma separated config files that override each other (files on the right override the left)
# each should start with file: or classpath:
# e.g. classpath:grouperText/grouper.text.en.us.base.properties, file:c:/temp/grouperText/grouper.text.en.us.properties
text.config.hierarchy = classpath:grouperText/grouper.text.fr.fr.base.properties, classpath:grouperText/grouper.text.fr.fr.properties

Change your browser to prefer French from France, (either wait a bit or bounce app server), and hit the UI and you should see your French text

Code changes

In the tomcat server.xml  Edit the <Connectors to have uri encoding of utf8, e.g.  <Connector  URIEncoding="UTF-8"

edu.internet2.middleware.grouper.ui.util.HttpContentType.java

FROM:

  TEXT_XML("text/xml"),
 
  /** text html content type */
  TEXT_HTML("text/html"),
 
  /** application json content type */
  APPLICATION_JSON("application/json"),

TO:

  TEXT_XML("text/xml;charset=utf-8"),
 
  /** text html content type */
  TEXT_HTML("text/html"),
 
  /** application json content type */
  APPLICATION_JSON("application/json;charset=utf-8"),

Insert some subjects in the DB/LDAP (if you don't already have some in an existing source, note, you need the default source enabled if you use this sql):

insert into subject (subjectId, subjectTypeId, name) values ("joñ", "person", "joñ");

insert into subjectattribute (subjectId, name, value, searchValue) values ("joñ", "description", "Joñ", "joñ");
insert into subjectattribute (subjectId, name, value, searchValue) values ("joñ", "loginid", "joñ", "joñ");
insert into subjectattribute (subjectId, name, value, searchValue) values ("joñ", "name", "Joñ", "joñ");
commit;

grouper/grouper-ui.base.properties:

 ###################################
 ## Internationalization
 ###################################

 # as of 2014/09/01 and Grouper 2.2.1+, this should be false
 convertInputToUtf8 = false

edu.internet2.middleware.grouper.j2ee.GrouperRequestWrapper:

FROM:

  public String getParameter(String name) {
 
    if (!this.multipart) {
      
      return this.wrapped.getParameter(name);
    }
 
    Object objectSubmitted = this.parameterMap.get(name);


TO:

  public String getParameter(String name) {
 
    if (!this.multipart) {
      String param = this.wrapped.getParameter(name);
      if (param != null && StringUtils.equals("GET", this.getMethod()) && TagUtils.mediaResourceBoolean("convertInputToUtf8", true)) {
        try {
          byte[] bytes = param.getBytes("ISO-8859-1");
          param = new String(bytes, "UTF-8");
        } catch (Exception e) {
          throw new RuntimeException(e);
        }
      }
      return param;
    }
 
    Object objectSubmitted = this.parameterMap.get(name);

edu.internet2.middleware.grouper.ui.GrouperUiFilter:

FROM:

   public void doFilter(ServletRequest servletRequest, ServletResponse response,
       FilterChain filterChain) throws IOException, ServletException {

     GrouperRequestWrapper httpServletRequest = null;
    
     try {
      
       httpServletRequest = new GrouperRequestWrapper((HttpServletRequest) servletRequest);


 TO:

   public void doFilter(ServletRequest servletRequest, ServletResponse response,
       FilterChain filterChain) throws IOException, ServletException {

     GrouperRequestWrapper httpServletRequest = null;
    
     try {
      
       servletRequest.setCharacterEncoding("UTF-8");
      
       httpServletRequest = new GrouperRequestWrapper((HttpServletRequest) servletRequest);


sdf

Testing

  • Create folder on new UI with special char
  • Create group in that folder with special char
  • Click on the folder breadcrumb to test is
  • Search for the folder in a folder combobox, select it
  • Put accented chars in the grouper text file and see those on the UI
  • Do the tests below too

Text on admin UI

custom/nav.properties

groups.action.show-summary=Group summáry

Looks like this on Group screen:



Text on Lite UI

Set this on custom/media.properties

ui-lite.link-from-admin-ui = true

login.ui-lite.show-link = true

custom/nav.properties

simpleMembershipUpdate.updateTitle=Group mêmbership update lite


Tooltip on the lite UI

custom/nav.properties

 tooltipTargetted.simpleMembershipUpdate.viewInAdminUi=Switçh to the Admin UI for a more complete set of features


Menu item lite

custom/nav.properties

simpleMembershipUpdate.advancedMenuShowMemberFilter=Séarch for member


Menu tooltip lite

custom/nav.properties

simpleMembershipUpdate.advancedMenuShowMemberFilterTooltip=Sèlecting this option will show a search box above the membership list where you can search for members in this group


Lite combo and submitting


See Also

Grouper Configuration Overlays

API Building and Configuration

 


 

  • No labels