From what I understand, in OSGi you can update jars at runtime without restarting the server. But jboss also has hot-deployment in which the full ear is updated and the server is still running.
So what would be the benefits of OSGi in an enterprise java project in jboss ?
I believe that the answer is the same as every OSGi use case: modularity and finer update granularity.
OSGi is far more than updating jars at runtime without restarting the server. From the perspective of your question, it is updating jars at runtime without restarting the application.
I admit I do not know the particular implementation of EAR hot deployment in JBoss AS, but in any case EAR updating cannot possibly be designed so to preserve the whole state of the application. The server is still running, but you essentially restart the application upon updating. The degree of such state loss really depends on how you design your application, but the fact remains that you are doing things monolitically.
With OSGi this is not the case: the application is made of a large set of bundles, each hopefully designed to handle a separate part of the functionality. This approach enables intra-application hot deployment, since the framework is designed to consider the effect that restarting any single jar brings to the application as a whole, and to let the other jars react appropriately. That offers the ability to preserve application state as much as possible.
Hence the benefits of an OSGi design in the Enterprise case is application liveness. No need to underline the importance of that. Truly, there are use cases where the application may be safely restarted. But OSGi in my opinion is the only really scalable and maintainable choice for Java EE nowadays. The fact that the most important application servers have moved (or are going to) to an OSGi runtime (and consequently giving OSGi application support) is proof of that.