JavaEE developer here.
Will these technologies be sufficient for complex and interactive UI, since I want to avoid Browser plug-ins and Applets? (OK, this is not directly a back-end question)
As performance (up to 25 clients might simultaneously request computational-expensive sub-routines), platform-independence (might run on Windows servers in future) and extensibility are important issues, I decided for Java. But would DJANGO or Ruby-on-Rails or PHP or something else be a sufficiently fast alternative to?
Using JavaEE, it is very easy to make extensible applications. You might want to do some reading on methods to handle this in the future, but for right now, I've always been told to worry about scalability when, and only when, scaling becomes a problem. Switching to another technology will provide you different solutions to your problem, but there are ways to solve this issue in any language you choose. If you go with the architectural choice to provide data via RESTful API on your backend, you can program an interface on any platform you wish, using whatever technology you'd like (web browser, iPhone, Android, command line, etc). If you are concerned about calculation speed on the back end, Java is going to give you an edge over PHP or Ruby.