SQL Server 2008 R2 versions prior to 10.50.2811.0 have a bug which causes encrypted JDBC connections to fail. This issue has been resolved in SQL Server 2008 R2 Service Pack 1, Cumulative Update 6 (See KB 2653857).
If the SQL Server instance does not force all connections to be encrypted (and you are okay with not encrypting the connection), you can uncheck the Connect using SSL box.
If the SQL Server 2008 R2 instance is setup to force encryption, or you do not want to turn off SSL, the only option is to upgrade to Service Pack 1, Cumulative Update 6 or greater.
How to find your version of SQL Server
Open Microsoft SQL Server Management Studio and select Help > About.
How to find if the server instance is set to force encryption:
Open Sql Server Configuration Manager > expand SQL Server Network Configuration and right click on Protocols for MSSQLSERVER > select Properties > select Flags tab.
If the value of ForceEncryption is set to Yes, all connections must be encrypted, otherwise, the server will reject the connection.