Seed data for spring roo application.

When developing a spring roo application it can useful to have a mechanism, that creates test data on server startup, so that you have some data for manual testing.

Here I will describe how to accomplice that.

Make a class that implements ApplicationListener.

@Component
public class DataSeeder implements ApplicationListener {

    private final Log log = LogFactory.getLog(DataSeeder.class);

    @Override
    @Transactional
    public void onApplicationEvent(ContextRefreshedEvent event) {
        if (event.getApplicationContext().getParent() == null) { // root context
            log.info("event.getApplicationContext() = " + event.getApplicationContext());
            String runMode = "server";
            try {
                Resource resource = new ClassPathResource("/env.properties");
                Properties props = PropertiesLoaderUtils.loadProperties(resource);
                runMode = props.getProperty("run.mode");
            } catch (IOException ignored) {
            }
            log.info("run.mode = " + runMode);
            if ("server".equals(runMode)) {
                // make seed data
                President president = new President();
                president.setName("Ron Paul");
                president.persist();
            }
        }
    }
}

The method onApplicationEvent is called once for each refreshed context. Since an application often has more than one context we must make sure that we only create seed data once. By checking if the context has a parent we can make sure that we only create seed data when the root context is refreshed.

Another issue is that we don’t want to create seed data when unit testing. So we have to find out if we are in test mode or on the server. In the test directory I made a properties file: test/resources/env.properties with one property:
run.mode=test
, this properties file is only copied in the classpath during test. So if we can read its property we should not create seed data. That’s it.

Writing a custom JPA UserDetailService

When using Spring Security in a ROO application, often there is a need for user management, where you can add, edit and delete users etc.. For that you make entity beans for users, roles, groups etc. – depending on your needs. Then you change some sql queries in the jdbc-user-service section of the applicationContext-security.xml file, so that Spring Security uses the same tables as the new entity beans. I would prefer the security to access the tables in the same manner as the rest of the application. Here I will describe how to make a custom UserDetailService, that uses the JPA entity beans in the login process.
For this I only use one entity bean for the users, I have 4 roles and I don’t need more, so they are of type enum, but it could just as well have been a @ManyToMany relation to a role entity. Here is the user entity:

public class SiteUser {

    @NotNull
    @Size(min = 3, max = 30)
    @Column(unique = true)
    private String username;

    @Size(max = 100)
    private String passwd;

    @NotNull
    private Boolean enabled;

    @ElementCollection
    private Set<SiteRole> roles = new HashSet<SiteRole>();

    }
}

You need to create a new class that implements UserDetailService. There is only one method in the interface loadUserByUsername and it gives the fully populated User. Spring recommends to return an immutable implementation of UserDetails, so I will the return the usual spring User. The SiteUser. FindSiteUsersByUsername method is a ROO generated finder.

public class JpaUserDetailService implements UserDetailsService {

    @Override
    public UserDetails loadUserByUsername(String username) throws 								UsernameNotFoundException, DataAccessException {
        TypedQuery<SiteUser> siteUsersQuery = SiteUser.findSiteUsersByUsername(username);
        List<SiteUser> siteUsers = siteUsersQuery.getResultList();
        if (siteUsers.isEmpty()) {
            throw new UsernameNotFoundException("Username " + username + " not found");
        }
        SiteUser siteUser = siteUsers.get(0);
        Set<GrantedAuthority> authorities = new HashSet<GrantedAuthority>();
        Set<SiteRole> siteRoles = siteUser.getRoles();
        System.out.println("siteRoles = " + siteRoles);
        for (SiteRole siteRole : siteRoles) {
            authorities.add(new GrantedAuthorityImpl(siteRole.name()));
        }
        return new User(siteUser.getUsername(), siteUser.getPasswd(), siteUser.getEnabled(),
                true, true, true, authorities);
    }
}

Now we only need to tell spring security to use the new UserDetailservice. In applicationContext-security.xml do the following.

    <authentication-manager alias="authenticationManager">
        <authentication-provider user-service-ref="customUserDetailsService">
            <password-encoder hash="sha-256"/>
        </authentication-provider>
    </authentication-manager>

    <beans:bean id="customUserDetailsService" class="dk.amfibia.cv.security.JpaUserDetailService"/>

In the first attempt I ran into a LazyInitializationException in the siteUser.getRoles()

This error is described here https://jira.springsource.org/browse/ROO-609. It should have been solved in ROO 1.0.2 but the problem seems to be still here in 1.1.3 – I left a comment in jira. As described in the jira, the solution is, in the web.xml, to move the filter mapping of springSecurityFilterChain below the filter-mapping of Spring OpenEntityManagerInViewFilter. Now it works.

Spring Roo Controller for Extjs4 data store with pagination, sorting, filtering and server side validation.

In this article I will describe a Spring Controller, that supports CRUD operations with server side sorting, filtering and pagination.
On the client side i have a model and a store, that handles all the communication with the server. The visual components consist of:
– Ext.grid.Panel with a Ext.PagingToolbar and ‘new’ and ‘delete’ buttons and a filter field.
– Ext.form.Panel with ‘save’ and ‘cancel’ buttons

I will not go into details about the ExtJs here but try to focus on the server side json rest controller. Most of Extjs code you can pickup in the ExtJs4 examples. I will though list the model and store here. As you can see the model has 3 fields: id, description and price.

Ext.define('Item', {
    extend: 'Ext.data.Model',
    fields: [
        {
            name: 'id',
            type: 'int',
            useNull: true
        },
        'description',
        {
            name: 'price',
            type: 'float',
            useNull: true
        }
    ]
});

And the store uses remote filtering and sorting – all done by the controller on the server. You can see more details about the filtering in one of my previous posts. And it uses a json rest proxy. I also listed the proxy exception handler, where you see how to make the form display the validation errors returned by the controller. The client side validation errors are displayed in the same way, only you get the errors like this:
form.updateRecord(newItem);
var errors = newItem.validate();

    var store = Ext.create('Ext.data.Store', {
        autoSync: true,
        model: 'Item',
        pageSize: 4,
        remoteSort: true,
        remoteFilter: true,
        proxy: {
            type: 'rest',
            url: '/cv/secure/items',
            format: 'json',
            reader: {
                type: 'json',
                root: 'data'
            }
            ,
            writer: {
                type: 'json'
            },
            afterRequest: proxyAfterRequest,
            listeners: {
                exception: proxyOnException
            }
        }
    });

    var proxyOnException = function(proxy, response, operation) {
        if (response.responseText) {
            var responseObj = Ext.decode(response.responseText);
            formPanel.getForm().markInvalid(responseObj.errors);
        }
    };

Now to the controller.
I have a Spring-Roo generated entity called Item with the same fields as the ExtJs model from before. The id field is hidden in a aspect file managed by Spring Roo.

   private String description;

    @Min(0L)
    private Double price;

The existing Rest Controller generated by Roo is not suitable for json with extjs, so I’m going to make a new rest controller for this purpose.
First lets look at the method that gets the list of items. It is the method that handles sorting, filtering and pagination.
This is an example of the query string parameters, that the ExtJs data store sends:

page:1
start:0
limit:4
sort:[{"property":"description","direction":"DESC"}]
filter:[{"property":"description","value":"r"}]

, and extjs expects to get a response like this:

{
	"total":4,
	"data":[
		{"price":3212.0,"id":3,"version":0,"description":"tgharga"},
		{"price":343.0,"id":4,"version":0,"description":"srsrg"},
		{"price":0.0,"id":2,"version":0,"description":"regerg"},
		{"price":6456.0,"id":5,"version":0,"description":"frtg"}
	]
}

Here is the method:

@RequestMapping(method = RequestMethod.GET)
    public void list(@RequestParam(value = "page", required = false) Integer page,
                     @RequestParam(value = "start", required = false) Integer start,
                     @RequestParam(value = "limit", required = false) Integer limit,
                     @RequestParam(value = "sort", required = false) String sorts,
                     @RequestParam(value = "filter", required = false) String filters,
                     Model model) {

        List<ItemSorting> sortList = ItemSorting.decodeJson(sorts);
        List<ItemFilter> itemFilterList = ItemFilter.decodeJson(filters);
        if (page != null && start != null && limit != null) {
            assert (start == (page - 1) * limit);
            model.addAttribute("total", Item.countItems(itemFilterList));
            model.addAttribute("data", Item.findItems(start, limit, sortList, itemFilterList));
        }
    }

It is quit straight forward. I made support classes, ItemSorting and ItemFilter, where the json is decoded. I will get back to those classes later.
The resulting list of items and the total number of filtered items are returned by the finder methods of the Item entity class, using jpa2 CriteriaQuery.

    public static List<Item> findItems(int firstResult, int maxResults, List<ItemSorting> itemSortings, List<ItemFilter> itemFilters) {
        CriteriaBuilder cBuilder = entityManager().getCriteriaBuilder();
        CriteriaQuery<Item> itemQuery = cBuilder.createQuery(Item.class);
        Root<Item> from = itemQuery.from(Item.class);
        // make predicates
        Predicate[] predicates = ItemFilter.makeFilterPredicates(from, itemFilters);
        // make orders
        List<Order> orders = ItemSorting.makeOrders(from, itemSortings);
        // get the items
        itemQuery.select(from).where(predicates).orderBy(orders);
        TypedQuery<Item> itemTypedQuery = entityManager().createQuery(itemQuery);
        return itemTypedQuery.setFirstResult(firstResult).setMaxResults(maxResults).getResultList();
    }

    public static Long countItems(List<ItemFilter> itemFilters) {
        CriteriaBuilder cBuilder = entityManager().getCriteriaBuilder();
        CriteriaQuery<Long> totalQuery = cBuilder.createQuery(Long.class);
        Root<Item> from = totalQuery.from(Item.class);
        // make predicates
        Predicate[] predicates = ItemFilter.makeFilterPredicates(from, itemFilters);
        // get total number of items.
        totalQuery.select(cBuilder.count(from));
        totalQuery.where(predicates);
        TypedQuery<Long> longTypedQuery = entityManager().createQuery(totalQuery);
        return longTypedQuery.getSingleResult();
    }

Following are the two helper classes ItemSorting and ItemFilter, where json from the query strings are decoded and the ordering and filtering is done for the finder methods.
First ItemSorting:

public class ItemSorting {

    private final String fieldName;
    private final String direction;

    public ItemSorting(String fieldName, String direction) {
        this.fieldName = fieldName;
        this.direction = direction;
    }

    public static List<ItemSorting> decodeJson(String json) {
        List<ItemSorting> sorts = new ArrayList<ItemSorting>();
        if (json != null) {
            List<HashMap<String,String>> sortList = new JSONDeserializer<ArrayList<HashMap<String,String>>>().deserialize(json);
            for (HashMap<String,String> sort : sortList) {
                String fieldName = sort.get("property");
                String direction = sort.get("direction");
                sorts.add(new ItemSorting(fieldName, direction));
            }
        }
        return sorts;
    }

    public static ArrayList<Order> makeOrders(Root<Item> from, List<ItemSorting> itemSortings) {
        ArrayList<Order> orders = new ArrayList<Order>();
        CriteriaBuilder cBuilder = Item.entityManager().getCriteriaBuilder();
        if (itemSortings != null && !itemSortings.isEmpty()) {
            for (ItemSorting itemSorting : itemSortings) {
                orders.add("DESC".equals(itemSorting.direction) ? cBuilder.desc(from.get(itemSorting.fieldName)) : cBuilder.asc(from.get(itemSorting.fieldName)));
            }
        }
        return orders;
    }

}

And here ItemFilter:

public class ItemFilter {

    private final String fieldName;
    private final String value;

    public ItemFilter(String fieldName, String value) {
        this.fieldName = fieldName;
        this.value = value;
    }

    public static List<ItemFilter> decodeJson(String json) {
        List<ItemFilter> itemFilters = new ArrayList<ItemFilter>();
        if (json != null) {
            List<HashMap<String,String>> filterList = new JSONDeserializer<ArrayList<HashMap<String,String>>>().deserialize(json);
            for (HashMap<String,String> filter : filterList) {
                String fieldName = filter.get("property");
                String direction = filter.get("value");
                itemFilters.add(new ItemFilter(fieldName, direction));
            }
        }
        return itemFilters;
    }

    public static Predicate[] makeFilterPredicates(Root<Item> from, List<ItemFilter> itemFilters) {
        Predicate[] predicates = new Predicate[0];
        if (itemFilters != null && !itemFilters.isEmpty()) {
            List<Predicate> predicateList = new ArrayList<Predicate>();
            for (ItemFilter itemFilter : itemFilters) {
                if ("description".equals(itemFilter.fieldName)) {
                    predicateList.add(descriptionPredicate(from, itemFilter, predicateList));
                }
            }
            predicates = predicateList.toArray(predicates);
        }
        return predicates;
    }

    private static Predicate descriptionPredicate(Root<Item> from, ItemFilter itemFilter, List<Predicate> predicateList) {
        Expression<String> path = from.get(itemFilter.fieldName);
        CriteriaBuilder cBuilder = Item.entityManager().getCriteriaBuilder();
        return cBuilder.like(path, "%" + itemFilter.value + "%");
    }

}

Now the controller is able to do sorting, filtering and pagination. Next we will make the controller able to create new items. In this case it receives some json data that it must decode and validate and then create a new item or return the validation errors.

The extjs data store sends this json data in the request body:

{"id":null,"description":"test","price":55}

When thing go well it expects a response like this:

{"message":"Created new Item", "data":{"price":55.0,"id":1,"version":0,"description":"test"}, "success":true}

, and if when the validation dont pass, the errors are returned like this:

{"success":false, "errors":{"price":"must be greater than or equal to 0"}} 

I tried different ways to get the json in the requestbody bound to item and have it validated at once, but it is currently not supported – see: https://jira.springsource.org/browse/SPR-6709
, I think when using the jackson marshaller for binding the json in the requestbody, valdation is not supported. So it must be done manually. To have the validator injected, declare a Validator and have it @Autowired.

   @RequestMapping(method = RequestMethod.POST)
    public void create(@RequestBody Item item, Model model) throws BindException {
//        validate
        final BindingResult result = new BeanPropertyBindingResult(item, "");
        ValidationUtils.invokeValidator(this.validator, item, result);
        if (result.hasErrors()) {
            model.addAttribute("success", false);
            model.addAttribute("errors", getFieldErrors(result));
        } else {
            item.persist();
            model.addAttribute("success", true);
            model.addAttribute("message", "Created new Item");
            model.addAttribute("data", item);
        }
    }

    private static Map<String, String> getFieldErrors(BindingResult result) {
        List<ObjectError> errors = result.getAllErrors();
        Map<String, String> errorMap = new HashMap<String, String>();
        for (ObjectError error : errors) {
            if (error instanceof FieldError) {
                FieldError fieldError = (FieldError) error;
                errorMap.put(fieldError.getField(), fieldError.getDefaultMessage());
            }
        }
        return errorMap;
    }

This was pretty straight forward and the last thing I will show is the update and delete methods, which are also quite simple.

    @RequestMapping(value = "/{id}", method = RequestMethod.PUT)
    public void update(@RequestBody Item item, Model model) throws BindException {
//        validate
        final BindingResult result = new BeanPropertyBindingResult(item, "");
        ValidationUtils.invokeValidator(this.validator, item, result);
        if (result.hasErrors()) {
            model.addAttribute("success", false);
            model.addAttribute("errors", getFieldErrors(result));
        } else {
                Item pItem = Item.findItem(item.getId());
                pItem.setDescription(item.getDescription());
                pItem.setPrice(item.getPrice());
                pItem.flush();

                model.addAttribute("success", true);
                model.addAttribute("message", "Item Updated");
                model.addAttribute("data", pItem);
        }
    }

    @RequestMapping(value = "/{id}", method = RequestMethod.DELETE)
    public void delete(@PathVariable("id") Long id, Model model) {
        Item item = Item.findItem(id);
        item.remove();
        model.addAttribute("success", true);
        model.addAttribute("message", "Item Deleted");
        model.addAttribute("data", item);
    }

Now the controller can do all the CRUD operations.

Filtering with delay

Filtering data can be done using a filter button that is clicked once the filter options are filled and the result will then show. This is the traditional way of doing it and it is the preferred way in many cases. It is also possible to omit the filter button and have the result updated automatically as the filter options change. In this case, when the user writes a text in a filter field, it is not desired to fire the filter event for every key press, especially when the filtering is done remotely on the server – better wait until the user has finished writing. The filtering should only occur once the filter options has not changed for a certain amount og time – like ½ second.
To accomplice this in ExtJs 4 we use a DelayedTask. It is called for every change in the filter options but it waits for some time until it fires the filter function. When a new change event from the filter options are reported to the DelayedTask, the previous reported change event is cancelled and the time is reset.

var filterTask = new Ext.util.DelayedTask();

var filterChange = function(field, newValue) {
        filterTask.delay(500, filter, this, [field, newValue]);
};

var filter = function(field, newValue) {
        store.filters.clear();
        store.currentPage = 1;
        store.filter({
                property: 'description',
                anyMatch: true,
                value   : newValue
        });
};

In this case there is only one field in the filter options and filterChange is called on the fields change event. The filter task is then called with a delay of 500 ms. The function filter is where the filtering is done. It is called by the filterTask 500 ms after the last change event.

First the existing filters are cleared. Do not use the stores clearFilter method for this because it will load the unfiltered data. Next we want the filtered result to start on page 1. Do not use the stores loadPage(1) for this because it will also load the unfiltered data. Last we apply the new filter with the filter method and it will load page 1 of the filtered data.

Categories: extjs4 Tags: , ,

Setting up ContentNegotioationViewResolver for using json with spring roo.

I want to make a controller, that supports crud via json. I will use it against an extjs4 data store.
The controllers that spring roo generates, can not be used for this, so I will make a rest controller by hand.

First I will use ContentNegotioationViewResolver to resolve the view, based on extensions. Resolving based on accept-header is more risky since there is not a consistent behavior among the different browsers.

This is the setup:

    <bean class="org.springframework.web.servlet.view.ContentNegotiatingViewResolver">
        <property name="mediaTypes">
            <map>
                <entry key="json" value="application/json"/>
            </map>
        </property>
        <property name="viewResolvers">
            <list>
                <bean class="org.springframework.web.servlet.view.BeanNameViewResolver"/>
                <bean class="org.springframework.web.servlet.view.UrlBasedViewResolver" id="tilesViewResolver">
                    <property name="viewClass" value="org.springframework.web.servlet.view.tiles2.TilesView"/>
                </bean>
            </list>
        </property>
        <property name="defaultViews">
            <list>
                <bean class="org.springframework.web.servlet.view.json.MappingJacksonJsonView" />
            </list>
        </property>
        <!--<property name="ignoreAcceptHeader" value="true" />-->
    </bean>

Make sure MappingJackson is in the class path. And ignoreAcceptHeader should not be set to true if you still want the roo-generated web pages to work – they use accept-header.

Now we are ready to make the json rest controller.

Cross field model validation in extjs4

I dont see an easy way to make validations where one field depends on another. I think they must have forgotten to support it. What is missing is the model instance passed to the validation method. Now it looks like this:
    function(config, value)
and we would like it to look like this
    function(config, value, model)
, then we would have access to all fields.

But we just need to override Ext.data.Model.validate() and pass the model instance to the validation function as an extra parameter:

valid = validators[type](validation, this.get(field), this);

Ext.override(Ext.data.Model, {
    validate: function() {
        var errors = Ext.create('Ext.data.Errors'),
                validations = this.validations,
                validators = Ext.data.validations,
                length, validation, field, valid, type, i;

        if (validations) {
            length = validations.length;

            for (i = 0; i < length; i++) {
                validation = validations[i];
                field = validation.field || validation.name;
                type = validation.type;
                valid = validators[type](validation, this.get(field), this);

                if (!valid) {
                    errors.add({
                        field  : field,
                        message: validation.message || validators[type + 'Message']
                    });
                }
            }
        }

        return errors;
    }
});

Now we have access to the model instance:

Ext.data.validations.range = function(config, value, model) {
    ....
    var someValue = model.get('someField')
    ....
}