What docs or links are there that would describe passed bug that turned in to vulnerabilities of spring security and how does one go about trying to hack into the framework?
I'm mainly interested in how to work around login and authorization of the spring security framework. Any technique that would help me to create automated penetration tests would be great help.
Unrelated side-note (would like to know your opinion on it though):
One feature that I would make the framework awesome is banning of IP addresses that exceeded daily request limit, or even better request limit per hour. So far I have been doing his on Cisco ASA or IPS units but it would be very useful to have application log of the events integrated together with spring security.
Any intel on this would be greatly appreciated.
XSS exploits are one of the most infamous, but, you've probably tried them.
Check the OWASP Top 10 (google please). You can try all of those.
Also, for blocking requests, that seems like a very good idea, until you realize the fact that raw sockets allow any even remotely experienced hacker to forge their IP. I guess you could validate the IP (because many times the script used to attack the page would not have a valid IP, as they would be randomly generated), but, websites are slow enough anyway, and as soon as you would do that, you'd still be vulnerable to slightly more sophisticated attacks that use IP databases.
Also, check out Metasploit, its VERY easy to use, and should get you on the road really quickly. You can also script penetration tests, so, once you deploy an update, just run a Ruby script and it should test it for holes.
Related
I have been working on an algorithm ( Not mine, I am just modifying it ) that predicts when to buy and sell on the FOREX market. I need to be able to open and close orders, dynamically update parameters of the orders ( such as stoploss, maximum stop etc. ) and receive real time tick data.
I have been researching for well over a week, and have no success.
The closest I have gotten is using JavoNet and Mt4 Api
I managed to import the DLL into java and use a MQL4 function, which was AccountBalance(), however this has returned 0.0, which was not the account balance, I messed around with the code and the settings on MT4 client but still no luck.
Q0: Can anyone please point me in the right direction?
I am new to automated FOREX trading but from what I understand there is a broker somewhere with a MT4 server and I connect to that server with my MT4 client on my windows machine.
Q1: If this is the case, do I need to make an API work with the server side instead of my client side?
All these DLL's I have tried so far have been used with the MT4 client software on my machine.
I have also been doing some reading on the FIX-Protocol and ZeroMQ.
Q2: Can these help me achieve my goal in any way (instead of creating some bridges between JAVA and MT4 DLL's)?
A0: yes, forget straight about REST and synchronous, blocking chains in FX-trading domain
A1: well, not a typical way. MetaTrader Server is a proprietary suite of systems on the Broker-side and theirs API are not disclosed to allow some 3rd party integrations against.
A2: FIX-Protocol is the industry standard LP-interfacing lingua franca. In case you have contracted relations with your institutional trading provider, incl. the FIX-Protocol GWY-port, this may provide you an A-level access to the Market and to integrate your trading tools against. If this is the case, forget about MT4 instrumentation, as prime-time cadences are far beyond the MT4 Terminal localhost processing architecture ( multiple events with a sub-millisecond TimeDOMAIN resolution are common, whereas MQL4 does not provide any direct support for multithreaded-concurrent / better parallel programme scheduling designs ). FIX-Protocol events are simply off-the picture above, being far left, "before" the graph starts from 1st [ms] column.
ZeroMQ may help liberate your further designs from MQL4 limitations. May like to read my other posts on distributed systems, where MQL4 / ZeroMQ / ML-AI-predictors / GPU-processing infrastructures appear.
Anyway:
Enjoy the Wild Worlds of MQL4/MQL5
Interested? May also like reading other MQL4, ZeroMQ distributed processing and low-latency trading posts
You can try MetaApi https://metaapi.cloud cloud service which provides REST API and WebSocket API access to both MetaTrader 4 and MetaTrader 5 accounts.
Official REST API documentation: https://metaapi.cloud/docs/client
SDKs: https://metaapi.cloud/sdks (javascript, python and Java SDKs are provided as per April 2021)
It supports reading account information, positions, orders, trade history, receiving quotes, and accessing market data.
The service also provides copy trading API https://metaapi.cloud/docs/copyfactory and API to calculate forex trading metrics on a MetaTrader account https://metaapi.cloud/docs/metastats.
I started to code an expert with MQL5, naturally on MT5 platform, and I must admit that the difficulty of managing the application along with the increase of its complexity is high. It's not only due to a missing garbage collector, that of course imposes the deletion of the new instances, but also because Java offers a set of powerful data structures and syntax that MQL5 naturally doesn't have. Last but not least, talking about the community and the third party libraries available, there's a light year of the distance between Java and MQL5. I.e. if I need to find a library for a JSON conversion on the Java side I find dozens of official and stable versions, in the MQL5 community I have found only rubbish that I had to modify myself.
So, after numerous failed tries on coding my expert in MQL5 (not a simple one of course), I decided to adopt a radical approach: coding an application, client-side MQL5, and server-side Java, that provides a Java facade for the MT5 platform. Same API, same basic events and so on. Even though I thought more than once that I was getting stuck in a blind path, I kept coding and eventually, I made it, obtaining a really solid result.
Naturally, the REST interface drastically reduces the performances, and each request, even with Tomcat and MT5 running in the same localhost, is in the order of milliseconds, not micros, but on the other side this reduces only the suitability of this architecture, it doesn't make it useless at all.
Strategies like scalpelling and every kind of high-frequency trading are not good for such kind of scenario, vice-versa every other strategy in the longer period, even if intraday's ones, can be implemented successfully without any cons.
Last but not least, it isn't necessary to use the WebRequest() MQL5 method to call any Servlet container, it is possible to import the wininet.dll from the OS (talking about Windows) and the strategy tester will work as if the strategy has been coded in MQL5, maybe just a little bit slower.
To sum up, I wouldn't be so sarcastic on the Java facade approach for the FX trading platforms, citing only the nude performances without contextualizing the overall scenario is a naive approach to face the argument.
If you need to send/receive synchronous message between MT4 and Java application, REST would be the best approach because fast response matters in this scenario. Message Queue solutions like ZeroMQ fits better in asynchronous solutions, so it won't help you. Once you choose REST approach, you can use MQL4 WebRequest() to call your Java application.
WebRequest isn't the end of the world, you can submit http requests from your EA using API, works even with Strategy Tester.
In order to collect the tick information and open, update or close orders, you can use mt4 server api.
please check this url.
http://mtapi.online/#overlappable-4
Maybe you will find what you want.
And then I have also mt4 server api. If you have any questions please update me.
What security framework do you use in your Java projects?
I used Spring Security and Apache Shiro and they both look immature.
Spring Security flaws:
no native support for permissions;
no ability to use explicitly in Java code (sometimes it's necessary);
too much focused on classic (non AJAX) web applications.
Apache Shiro flaws:
bugs in final release (like the problem with Spring integration);
no support for OpenID and some other widely used technologies;
performance issues reported.
There is also lack of documentation for both of them.
Maybe most of the real projects develop their own security frameworks?
As for Apache Shiro:
I'm not sure why you've listed the things you did:
Every project in the world has release bugs, without question. The big key here however is that Shiro's team is responsive and fixes them ASAP. This is not something to evaluate a framework on, otherwise you'd eliminate every framework, including any you write yourself.
OpenID support will be released shortly in Shiro 1.2 - maybe a month out?.
What performance issues? No one has ever reported performance issues to the dev list, especially since the caching support in Shiro is broad and first-class. Without clarifications or references, this comes across as FUD.
Documentation now is really good actually - some of the best in Open Source that I've seen lately (it was re-worked 2 weeks ago). Do you have specific examples of where it falls short for you?
I'd love to help, but your concerns are generalizations that aren't supported by references or concrete examples. Maybe you could represent specific things that your project needs that you've fail to accomplish thus far?
Apache Shiro continues to be the most flexible and easiest to understand security framework for Java and JVM languages there is - I doubt you'll find better.
But, above all, and I mean this with all sincerity, please don't write your own security framework unless you plan on putting a ridiculous amount of time into it. Nearly every company I've ever seen that tries to do this themselves fails miserably. It is really hard to get 'right' (and secure). Trust me - after writing one for 8 years, that's one thing I'm absolutely sure of :)
Anyway, feel free to join the Shiro user list and you're sure to find that the community is happy and willing to work through whatever issues you may have. You'll find that we take care of the people that ask questions and do our best to help out.
HTH!
My current projects use SpringSecurity and involve doing all three things you claim to be flaws in SpringSecurity:
The projects implement fine-grained access rules that go beyond simple ROLEs, and variously involve state of domain objects, extra request parameters, and so on. These are implemented using custom "access policy objects" that get called within my MVC controllers. However, access check failures are handed back to SpringSecurity by throwing the relevant exception. (These could have been implemented as standard SpringSecurity method-level interceptors, but the checks typically involve examining domain objects.)
The projects support both web and AJAX access, and deal with access failures differently for the two cases. This is done by writing some custom Authentication entrypoint components for SpringSecurity that choose between different authentication behaviors depending on the request URL, etc.
In other words, it can be done ...
Having said that, I agree with you on a couple of points:
It is not easy to wire this up kind of thing. I kept on running into roadblocks when using the <http> element and its associated configurer. Like ... you want it to use a different version of component X. But to do that you have to replace Y, Z, P and Q as well.
The documentation is really sparse, and not helpful if you are trying to do something out of the ordinary.
Andrey, I think this answer comes too late to be helpful to you; it is intended for those who land on this thread later and I hope it helps.
My company recently released as open source, OACC, an advanced Java Application Security Framework. OACC is designed for systems that require up to object-level security granularity.
OACC provides a high performance API that provides permission based authorization services. In a nutshell, OACC allows your application to enforce security by answering the question: Is entity ‘A’ allowed to perform action ‘p’ on entity ‘B’?
One of the key abstractions in OACC is a resource. A resource serves as the placeholder in OACC for the object in the application domain that needs to be secured. Both the actors (e.g. users, processes) and the objects being secured (e.g. documents, servers) are represented as resources in OACC. The application domain objects that are actors, or are secured, simply store the resource id to the associated resource.
The resource abstraction allows OACC, unlike other major security frameworks, to provide a rich API that manages permissions between resources. OACC persists security relationships in RDBMS tables (DB2, Oracle, MS-SQLServer and PostgreSQL are currently supported).
For more information please check out the project website: http://oaccframework.org
We use a layered security in one of our projects. The layers are the following:
HTTPS as protocol (Apache-AJCConnectors-TomcatServlets)
Only binary objects transferred between client and servlet
Individual elements in the passed objects (either way) are encrypted
Encryption key is dynamic, set up during initial handshaking, valid for 1 session
Conceptually, the security consists of the encryption key, encryption algorithm and the data on which it is applied. We make sure that more than 1 of the 3 is never passed simultaneously during a communication. Hope that helps. Regards, - M.S.
I've used tuckey's UrlRewriteFilter in small projects, but I'm hesitant to use such a thing in a production environment that could touch tens of thousands of paying customers (it feels kludge-y). Is it fine to use a rule-based rewriting engine in production, and what are some alternatives I could use for clean URLs?
We're using the UrlRewriteFilter by Tuckey in our production environment without any noticeable issues or performance downfalls. Our services are heavily used with more then 10k hits per sec.
If you're using UrlRewrite just to process RESTful URLs - think about switching to Spring 3.0 (http://blog.springsource.com/2009/03/08/rest-in-spring-3-mvc/).
Also, consider using JAX-RS but I have no extensive knowledge about it's performance vs Spring.
Yes, it is fine. A lot of large sites are doing it, in one way or another.
I've used URL rewriting in mid-scale sites (10-20k visitors/day) and have never found it to be a bottleneck. I haven't used the reqriter you mention so there's a chance it may prove problematic.
In general, unless you've got some REALLY convoluted rules, the overhead of rewriting is going to be negligible compared to say opening a database connection.
There are also benefits to the user in terms of usability/remembering URLs and (also friendly URLs seem to make users feel more confident). It's also nicer when you're digging through error logs :)
Here's a use case:
I have a desktop application (built using Eclipse RCP) which on start, pops open a dialog box with 'UserName' and 'Password' fields in it. Once the end user, inputs his UserName and Password, a server is contacted (a spring remote-servlet, with the client side being a spring httpclient: similar to the approaches here.), and authentication is performed on the server side.
A few questions related to the above mentioned scenario:
If said this authentication service were to go down, what would be the best way to handle further proceedings? Authentication is something that I cannot do away with. Would running the desktop client in a "limited" mode be a good idea? For instance, important features/menus/views will be disabled, rest of the application will be accessible?
Should I have a back up authentication service running on a different machine, working as a backup?
What are the general best-practices in this scenario? I remember reading about google gears and how it would let you edit and do stuff offline - should something like this be designed?
Please let me know your design/architectural comments/suggestions. Appreciate your help.
The simple answer is: Don't let the authentication service go down!
Make sure your authentication service is running in a clustered, load balanced environment behind a virtual IP. That way, you can avoid downtime in the event that one of the individual servers goes down. This goes for not only the service itself, but any data sources that it relies on.
Obviously no system is completely fail-safe, but you should be able to get your uptime close enough to 100% that there is no need to build a "limited" mode for the desktop client.
Should I have a back up authentication service running on a different machine, working as a backup?
Yes! This would be the best solution. The issue should IMO be dealt with on the network/infrastructure level, not on the client.
If there are useful parts of the application that could still function with no network access (e.g. router down, NIC goes pop), option 1 might be considered. Offset the amount of work necessary against how likely this is and how critical your app is.
If said this authentication service
were to go down, what would be the
best way to handle further
proceedings? Authentication is
something that I cannot do away with.
Would running the desktop client in a
"limited" mode be a good idea? For
instance, important
features/menus/views will be disabled,
rest of the application will be
accessible?
Running the desktop client in a limited way is a very good idea. Imagine if you were unable to write an email, perpare attachments, or do anything in an email client if you were not logged in. A good user experience demands the ability to work offline.
Should I have a back up authentication
service running on a different
machine, working as a backup?
This has been answers very well by others already although I don't agree entirely with dbyrne. Even though all your networks and servers may be running fine, downtime is inevitable and the communication between the desktop client and the server will not always be perfect.
If said this authentication service were to go down, what would be
the best way to handle further
proceedings? Authentication is
something that I cannot do away with.
Would running the desktop client in a
"limited" mode be a good idea? For
instance, important
features/menus/views will be disabled,
rest of the application will be
accessible?
Is the client useful without the server? Are there things the user can do? If so, do you want the user to be able to do these things without authenticating? That's the answer to your question.
It isn't clear what you mean when you say: "Authentication is something that I cannot do away with." what you mean. Do you mean that there are some features that require the user to be authenticated, or that it is a requirement imposed by someone else, or ? (Why can't you do away with it?)
Should I have a back up authentication service running on a
different machine, working as a
backup?
How useful is your client in the situation above? If it is very useful, then you can base this decision and how much to spend on maintaining a backup server on how valuable the authenticated features are alone.
If your application is useless without authentication, then base your decision as to how much to invest in a backup authentication server on how much it costs you when your users cannot authenticate.
What are the general best-practices in this scenario? I
remember reading about google gears
and how it would let you edit and do
stuff offline - should something like
this be designed?
If there is a way to keep useful data offline, I think that's a good idea, but I am biased against keeping my information in the cloud where I can't control it or back it up. It will cost time and implicitly money to develop the ability to do both online and offline versus just one of the two. This is a judgement call on how valuable the application is to your users when offline.
I'm building a VERY simple web hosting service that will cater to the ma and pa type small business.
Now my dilemma is whether I should built it from scratch or use an existing CMS. The CMS needs to be customisable, as I wish to build my own client. I only want the user to have to put a title and content. Everything else will be hidden from the end user.
Later on I want to be able to associate my own custom built web applications to a given page, so the CMS needs to be able to cater for that.
The other requirement is that it needs to be Java based (Groovy acceptable).
Any ideas?
Even a "simple" CMS is a fairly complicated application once you start doing things like user authentication, security, scalability, etc. There are many many things you can get wrong. The major CMSes out there suffer from lots of maintenance problems and security bugs and there are a fair number of talented people working on them. If you think your CMS will be different, I suggest that it will not be. You will need to update and maintain it constantly. If you use an off-the-shelf solution you should be able to benefit from the work those other developers are doing. If you want to write your own code, consider customizing an off-the-shelf CMS or contributing new features or bug fixes.
Own CMS:
Total control over the features
Low cost
Easy for you to understand
Only you understand it and can fix it
No bug fixes from other developers
Off-the-shelf CMS:
You can simply install it, then concentrate on adding value
Steeper learning curve
Lots of hosted solutions and online help
Lots of people can admin it if you're not around
Bug fixes and security updates are released by the vendor
More limited in terms of customization, etc
Someone needs to keep on top of the updates and install them, or else the customer might be victim to a worm such as the WordPress worms. With your own CMS it's less likely that someone will create a worm just for you. (But your customers are still at risk of other security problems).
Consider the typical security issues that face every website faces: XSS, CSRF, SQL Injection, configuration errors, loose security, session hijacking, parameter validation errors, race conditions, etc. You need to handle all those cases, but the CMS vendors are already doing that for you.
As for your Java requirement, the Resin web server ships with a PHP interpreter which might allow you to deploy a PHP app in Java should you choose a PHP CMS. It should also be possible to port the PHP engine to another servlet container in a few hours (I think it's GPL).
You should definitely learn an existing CMS. As much as creating one from scratch will fit your needs. An existing CMS with a live community (such as drupal or django) will have sold foundations, based on the accumulated experience of others.
Doing so will help skip the common mistakes of creating a CMS, and will allow you to focus on functionality and making your product, versus building a framework.
Check out Alfresco Community Edition, it's java based and it's the first one to be compliant with CMIS 1.0. You can build up a web client/application on it. It has ftp, webdav, cifs, http interface.
Liferay? You'll be up and running in less than 5 minutes.
Java based, Supports Groovy
If you want to diy, take a look at CMIS. (Forget it, don't diy)
I can't imagine there isn't an existing CMS that would fit those requirements. Won't do the research for you though ;-)
If you want to figure out some Java EE technology aspects or advantages/disadvantages it is better to write your own CMS just for practice.
But before you do this go through some of the existing CMS to create list of pros and cons.
All existing CMS/frameworks/whatever can't fit well your own problem but instead they are often too big to cover as many cases as the can.
One more thing to think about. Do you want your CMS to work great at Google App Engine?