| Include Page | ||||
|---|---|---|---|---|
|
This page presents information on Grouper and internationalization, handling non-English characters, UTF-8, accents, etc.
Grouper and non-English characters
Grouper works with non-English characters, but the environment must be setup correctly for this to happen. Your database must support it, your Java settings must be set correctly, your Tomcat settings must be set correctly. In Grouper 2.2.1 patch grouper_v2_2_1_api_patch_9 there are checks at startup. Grouper patches are listed on the Release Notes page.
There are checks at startup to make sure the non-Tomcat settings are correct.
Also, the default for grouper.base.properties for file encoding was changed. If you don't need non-English chars and you are worried about the risk of this change, please change it in grouper.properties
| Code Block |
|---|
# when reading writing files from util classes, this is encoding (was ISO-8859-1)
grouper.default.fileEncoding = UTF-8 |
These settings will detect if your database and file system can handle UTF-8, and if transactions, and case sensitivity are handled correctly. You can get a success message if you are debugging, but this defaults to off. These will run in another thread so they don't affect start time (optionally default to true)
| Code Block |
|---|
# if should check database and utf in new thread
configuration.checkDatabaseAndUtf.inNewThread = true
# if grouper should check to see if the database has case sensitive selects
configuration.detect.db.caseSensitive.problems = true
configuration.display.db.caseSensitive.success.message = false
# if grouper should check to see if utf-8 works on startup in files
configuration.detect.utf8.file.problems = true
# if grouper should check to see if utf-8 works on startup in the database
configuration.detect.utf8.problems = true
configuration.display.utf8.success.message = false
# if grouper in the utf8 check will check to see if grouper supports transaction
configuration.detect.db.transaction.problems = true
configuration.display.transaction.success.message = false |
If there is a problem with start, you will see a message like this, but it will not cause grouper not to start:
| Code Block |
|---|
Error: Cannot properly read UTF string from database: '???????', make sure your database has UTF tables and perhaps a hibernate.connection.url in grouper.hibernate.properties |
Note, it will try to give suggestions about what the problem could be.
Grouper and UTF-8
| Info |
|---|
utf8mb4 is not recommended because the database engines in MySQL/MariaDB that support transactions have limitations on the length of prefixes for indices. For InnoDB and XtraDB, the prefix is limited to 767 bytes. When using utf8mb4, where 4 bytes are used instead of 2, the prefixes used for some Grouper tables are too long. |
For Grouper to handle UTF-8 characters:
Make sure java starts with this setting: -Dfile.encoding=UTF-8 for example, in tomcat, put something like this in your startup script:
Code Block export JAVA_OPTS="-server -Xms50M -Xmx200M -XX:MaxPermSize=95M -Dfile.encoding=UTF-8"Make sure URIEncoding is set to UTF-8 in the server.xml in tomcat in the Connector element, here is an example
Code Block <Connector port="8111" protocol="AJP/1.3" request.tomcatAuthentication="false" URIEncoding="UTF-8" />- Make sure your database tables are using UTF-8. For Mysql you might want to use utf8mb4. If not, e.g. for MySQL, you might need to SQL dump, delete the DB, create with default UTF-8 bin collation, and import the SQL.
Make sure your connect string to the database is configured to use UTF-8 (if applicable). For MySQL, here is an example (see the stuff after the question mark). Note, in Windows MySQL this isn't necessary
Code Block hibernate.connection.url = jdbc:mysql://localhost:3306/grouper_v2_2?CharSet=utf8&useUnicode=true&characterEncoding=utf8
This document relates to Grouper 1.6 and internationalization. Note,
Note, my environment is windows, mysql, tomcat.
Grouper v2.2+ translations
If you want to add a localization to the grouper Grouper UI text (example for frenchFrench):
Add this to your grouper-ui.properties (index must increment from the base file:
| Code Block |
|---|
# language for this bundle
grouper.text.bundle.1.language = fr
# country for this bundle
grouper.text.bundle.1.country = fr
# filename in the package grouperText that is before the .base.properties, and .properties
grouper.text.bundle.1.fileNamePrefix = grouperText/grouper.text.fr.fr
|
...
Have this at the top of grouper.text.fr.fr.base.properties
| Code Block |
|---|
########################################
## Config chaining hierarchy
########################################
# comma separated config files that override each other (files on the right override the left)
# each should start with file: or classpath:
# e.g. classpath:grouperText/grouper.text.en.us.base.properties, file:c:/temp/grouperText/grouper.text.en.us.properties
text.config.hierarchy = classpath:grouperText/grouper.text.fr.fr.base.properties, classpath:grouperText/grouper.text.fr.fr.properties
|
Change your browser to prefer french French from franceFrance, (either wait a bit or bounce app server), and hit the UI and you should see your french French text
Code changes
In the tomcat server.xml Edit the <Connectors to have uri encoding of utf8, e.g. <Connector URIEncoding="UTF-8"
edu.internet2.middleware.grouper.ui.util.HttpContentType.java
| Code Block |
|---|
FROM:
TEXT_XML("text/xml"),
/** text html content type */
TEXT_HTML("text/html"),
/** application json content type */
APPLICATION_JSON("application/json"),
TO:
TEXT_XML("text/xml;charset=utf-8"),
/** text html content type */
TEXT_HTML("text/html"),
/** application json content type */
APPLICATION_JSON("application/json;charset=utf-8"),
|
Insert some subjects in the DB/LDAP (if you dont don't already have some in an existing source, note, you need the default source enabled if you use this sql):
| Code Block |
|---|
insert into subject (subjectId, subjectTypeId, name) values ("joñ", "person", "joñ");
insert into subjectattribute (subjectId, name, value, searchValue) values ("joñ", "description", "Joñ", "joñ");
insert into subjectattribute (subjectId, name, value, searchValue) values ("joñ", "loginid", "joñ", "joñ");
insert into subjectattribute (subjectId, name, value, searchValue) values ("joñ", "name", "Joñ", "joñ");
commit;
|
grouper/grouper-ui.base.properties:
| Code Block |
|---|
###################################
## Internationalization
###################################
# as of 2014/09/01 and Grouper 2.2.1+, this should be false
convertInputToUtf8 = false
|
edu.internet2.middleware.grouper.j2ee.GrouperRequestWrapper:
| Code Block |
|---|
FROM:
public String getParameter(String name) {
if (!this.multipart) {
return this.wrapped.getParameter(name);
}
Object objectSubmitted = this.parameterMap.get(name);
TO:
public String getParameter(String name) {
if (!this.multipart) {
String param = this.wrapped.getParameter(name);
if (param != null && StringUtils.equals("GET", this.getMethod()) && TagUtils.mediaResourceBoolean("convertInputToUtf8", true)) {
try {
byte[] bytes = param.getBytes("ISO-8859-1");
param = new String(bytes, "UTF-8");
} catch (Exception e) {
throw new RuntimeException(e);
}
}
return param;
}
Object objectSubmitted = this.parameterMap.get(name);
|
edu.internet2.middleware.grouper.ui.GrouperUiFilter:
| Code Block |
|---|
FROM:
public void doFilter(ServletRequest servletRequest, ServletResponse response,
FilterChain filterChain) throws IOException, ServletException {
GrouperRequestWrapper httpServletRequest = null;
try {
httpServletRequest = new GrouperRequestWrapper((HttpServletRequest) servletRequest);
TO:
public void doFilter(ServletRequest servletRequest, ServletResponse response,
FilterChain filterChain) throws IOException, ServletException {
GrouperRequestWrapper httpServletRequest = null;
try {
servletRequest.setCharacterEncoding("UTF-8");
httpServletRequest = new GrouperRequestWrapper((HttpServletRequest) servletRequest);
|
sdf
Testing
- Create folder on new UI with special char
- Create group in that folder with special char
- Click on the folder breadcrumb to test is
- Search for the folder in a folder combobox, select it
- Put accented chars in the grouper text file and see those on the UI
- Do the tests below too
Text on admin UI
custom/nav.properties
| Code Block |
|---|
groups.action.show-summary=Group summáry
|
Looks like this on Group screen:
...
...
Text on Lite UI
Set this on custom/media.properties
| Code Block |
|---|
ui-lite.link-from-admin-ui = true
login.ui-lite.show-link = true
|
custom/nav.properties
| Code Block |
|---|
simpleMembershipUpdate.updateTitle=Group mêmbership update lite
|
...
...
Tooltip on the lite UI
custom/nav.properties
| Code Block |
|---|
tooltipTargetted.simpleMembershipUpdate.viewInAdminUi=Switçh to the Admin UI for a more complete set of features
|
...
...
Menu item lite
custom/nav.properties
| Code Block |
|---|
simpleMembershipUpdate.advancedMenuShowMemberFilter=Séarch for member
|
...
...
Menu tooltip lite
custom/nav.properties
| Code Block |
|---|
simpleMembershipUpdate.advancedMenuShowMemberFilterTooltip=Sèlecting this option will show a search box above the membership list where you can search for members in this group
|
...
...
Lite combo and submitting
...
sdf
See Also
Grouper Configuration Overlays
API Building and Configuration
...