Can an Encryption Seed/Algorithm be Reverse-Engeered from Compiled Code?

Question

I've been learning a lot about encryption lately, and I'm thinking about using it in some of my Java networking applications. My idea is to have a pseudorandom number generator installed on the client application and the server application, and they will both pass the same seed to that algorithm so that they generate the same encryption keys. My question: If someone reverse-engineered the compiled code, would it be possible for them to find the algorithm and/or seed? If so, is there a way to defend against this? I develop with java and some similar questions that I've seen haven't been very specific.

Answer 1

In general, yes. But you should give more info about your idea, and read more about the state of the art of encryption.

If your seed is always the same, hardcoded, the pseudorandom generator will always give the same sequence of numbers and your keys won't be secure.

The algorithm is usually public, and the keys shared with asymetric techniques (see Diffie-Hellman as an introduction), and then generate a symetric key at runtime for speed at encryption of the messages.

Answer 2

So you decided to employ cryptography. That is a very good and valuable first insight, that can not be honoured enough.

In theory: Yes. If your computer can execute it, it is deterministic and finite and thus the code's inner working can be examined and reverse-engineered.

In practise: It takes quite a lot of time and effort to take apart a simple Hello World example. Reverse-engineering more complex code requires a lot of time, effort and knowledge. If your "product" does not deal with any significant amount of or valuable information it becomes more unlikely that someone will actually spend time trying to take the thing apart.

However, since you consider your information valuable enough that you want to encrypt it, always assume that someone else will too - and has the required resources (time, knowledge, and/or money…) available to do it.

A few pieces of advice: Since you obviously started looking into cryptography, you already noticed that it is hard. And it is even harder to get it right. And it is still harder to get it secure and resilient.

• A good cryptographic algorithm does not require to be obfuscated. Even if all its inner workings are well-known and understood, it provides reliable security. Security through obscurity is a common misconception in beginners and "experts" alike. A lot of people/companies have fallen into this trap and the damage usually was severe.

• Cryptography is all or nothing. If you have a small bug in your code, all your encrypted information are de-facto no longer secret. If your random number generator has a bug, your encryption is nothing but paper wrapping around a present… Even famous libraries like OpenSSL always need a ton of bug-fixing, maintenance and reviews.

• Don't invent your own cryptography for use in a live software. You may do it on paper for learning and you may implement some examples for playing around with it. However, It took a huge amount of people a lot of time to design, prove (mathematically), implement our currently used algorithms - for a good reason.

• Do research already existing implementations for well-known algorithms and try to figure out if they are well-maintained and up-to-date. Learn to understand them and to properly use them. (Common mistake #2: programmer unable to properly read and obey the library documentation)

Conclusion: Do play around with cryptography to get a feel for it. However, do use well established implementations in the "real" world, unless you really hate your users. Dealing with cryptographic libraries/frameworks is hard enough. Get into the details if you are willing to spend a considerable amount of your life with it.

Oh, and reverse-engineering? Stop caring about that. If your attacker wants to do it, he will. But if your security implementation is good, he won't benefit from the reverse-engineering at all.