Merge branch 'release/0.20.0'

Update from Weblate

.idea/dictionaries/bmarty.xml

@@ -25,6 +25,7 @@
       <w>signup</w>
       <w>ssss</w>
       <w>threepid</w>
+      <w>unwedging</w>
     </words>
   </dictionary>
 </component>

CHANGES.md

@@ -1,4 +1,24 @@
-Changes in RiotX 0.19.0 (2020-XX-XX)
+Changes in RiotX 0.20.0 (2020-05-15)
+===================================================
+
+Features ✨:
+ - Add Direct Shortcuts (#652)
+
+Improvements 🙌:
+ - Invite member(s) to an existing room (#1276)
+ - Improve notification accessibility with ticker text (#1226)
+ - Support homeserver discovery from MXID (DISABLED: waiting for design) (#476)
+
+Bugfix 🐛:
+ - Fix | Verify Manually by Text crashes if private SSK not known (#1337)
+ - Sometimes the same device appears twice in the list of devices of a user (#1329)
+ - Random Crashes while doing sth with cross signing keys (#1364)
+ - Crash | crash while restoring key backup (#1366)
+
+SDK API changes ⚠️:
+ - excludedUserIds parameter added to the UserService.getPagedUsersLive() function
+
+Changes in RiotX 0.19.0 (2020-05-04)
 ===================================================
 
 Features ✨:
@@ -7,6 +27,7 @@ Features ✨:
  - Cross-Signing | Verify new session from existing session (#1134)
  - Cross-Signing | Bootstraping cross signing with 4S from mobile (#985)
  - Save media files to Gallery (#973)
+ - Account deactivation (with password only) (#35)
 
 Improvements 🙌:
  - Verification DM / Handle concurrent .start after .ready (#794)
@@ -22,6 +43,14 @@ Improvements 🙌:
  - Emoji Verification | It's not the same butterfly! (#1220)
  - Cross-Signing | Composer decoration: shields (#1077)
  - Cross-Signing | Migrate existing keybackup to cross signing with 4S from mobile (#1197)
+ - Show a warning dialog if the text of the clicked link does not match the link target (#922)
+ - Cross-Signing | Consider not using a spinner on the 'complete security' prompt (#1271)
+ - Restart broken Olm sessions ([MSC1719](https://github.com/matrix-org/matrix-doc/pull/1719))
+ - Cross-Signing | Hide Use recovery key when 4S is not setup (#1007)
+ - Cross-Signing | Trust account xSigning keys by entering Recovery Key (select file or copy) #1199
+ - E2E timeline decoration (#1279)
+ - Manage Session Settings / Cross Signing update (#1295)
+ - Cross-Signing | Review sessions toast update old vs new (#1293, #1306)
 
 Bugfix 🐛:
  - Fix summary notification staying after "mark as read"
@@ -36,9 +65,13 @@ Bugfix 🐛:
  - Render image event even if thumbnail_info does not have mimetype defined (#1209)
  - RiotX now uses as many threads as it needs to do work and send messages (#1221)
  - Fix issue with media path (#1227)
+ - Add user to direct chat by user id (#1065)
+ - Use correct URL for SSO connection (#1178)
+ - Emoji completion :tada: does not completes to 🎉 like on web (#1285)
+ - Fix bad Shield Logic for DM (#963)
 
 Translations 🗣:
- -
+ - Weblate now create PR directly to RiotX GitHub project
 
 SDK API changes ⚠️:
  - Increase targetSdkVersion to 29

CONTRIBUTING.md

@@ -13,6 +13,24 @@ Dedicated room for RiotX: [![RiotX Android Matrix room #riot-android:matrix.org]
 Please set the "hard wrap" setting of Android Studio to 160 chars, this is the setting we use internally to format the source code (Menu `Settings/Editor/Code Style` then `Hard wrap at`).
 Please ensure that your using the project formatting rules (which are in the project at .idea/codeStyles/), and format the file before committing them.
 
+### Template
+
+An Android Studio template has been added to the project to help creating all files needed when adding a new screen to the application. Fragment, ViewModel, Activity, etc.
+
+To install the template (to be done only once):
+- Go to folder `./tools/template`.
+- Run the script `./configure.sh`.
+- Restart Android Studio.
+
+To create a new screen:
+- First create a new package in your code.
+- Then right click on the package, and select `New/New Vector/RiotX Feature`.
+- Follow the Wizard, especially replace `Main` by something more relevant to your feature.
+- Click on `Finish`.
+- Remainning steps are described as TODO in the generated files, or will be pointed out by the compilator, or at runtime :)
+
+Note that if the templates are modified, the only things to do is to restart Android Studio for the change to take effect.
+
 ## Compilation
 
 For now, the Matrix SDK and the RiotX application are in the same project. So there is no specific thing to do, this project should compile without any special action.

docs/notifications.md

@@ -38,10 +38,10 @@ When the client receives the new information, it immediately sends another reque
 This effectively emulates a server push feature.
 
 The HTTP long Polling can be fine tuned in the **SDK** using two parameters:
-* timout (Sync request timeout)
+* timeout (Sync request timeout)
 * delay (Delay between each sync)
 
-**timeout** is a server paramter, defined by:
+**timeout** is a server parameter, defined by:
 ```
 The maximum time to wait, in milliseconds, before returning this request.`
 If no events (or other data) become available before this time elapses, the server will return a response with empty fields.

docs/signin.md

@@ -57,7 +57,7 @@ We get credential (200)
 
 ```json
 {
-  "user_id": "@benoit0816:matrix.org",
+  "user_id": "@alice:matrix.org",
   "access_token": "MDAxOGxvY2F0aW9uIG1hdHREDACTEDb2l0MDgxNjptYXRyaXgub3JnCjAwMTZjaWQgdHlwZSA9IGFjY2VzcwowMDIxY2lkIG5vbmNlID0gfnYrSypfdTtkNXIuNWx1KgowMDJmc2lnbmF0dXJlIOsh1XqeAkXexh4qcofl_aR4kHJoSOWYGOhE7-ubX-DZCg",
   "home_server": "matrix.org",
   "device_id": "GTVREDALBF",
@@ -128,6 +128,8 @@ We get the credentials (200)
 }
 ```
 
+It's worth noting that the response from the homeserver contains the userId of Alice.
+
 ### Login with Msisdn
 
 Not supported yet in RiotX

gradle.properties

@@ -8,7 +8,7 @@
 # The setting is particularly useful for tweaking memory settings.
 android.enableJetifier=true
 android.useAndroidX=true
-org.gradle.jvmargs=-Xmx1536m
+org.gradle.jvmargs=-Xmx8192m
 # When configured, Gradle will run in incubating parallel mode.
 # This option should only be used with decoupled projects. More details, visit
 # http://www.gradle.org/docs/current/userguide/multi_project_builds.html#sec:decoupled_projects

matrix-sdk-android-rx/src/main/java/im/vector/matrix/rx/RxRoom.kt

@@ -28,6 +28,7 @@ import im.vector.matrix.android.api.session.room.send.UserDraft
 import im.vector.matrix.android.api.session.room.timeline.TimelineEvent
 import im.vector.matrix.android.api.util.Optional
 import im.vector.matrix.android.api.util.toOptional
+import io.reactivex.Completable
 import io.reactivex.Observable
 import io.reactivex.Single
 
@@ -95,6 +96,10 @@ class RxRoom(private val room: Room) {
     fun liveNotificationState(): Observable<RoomNotificationState> {
         return room.getLiveRoomNotificationState().asObservable()
     }
+
+    fun invite(userId: String, reason: String? = null): Completable = completableBuilder<Unit> {
+        room.invite(userId, reason, it)
+    }
 }
 
 fun Room.rx(): RxRoom {

matrix-sdk-android-rx/src/main/java/im/vector/matrix/rx/RxSession.kt

@@ -31,6 +31,8 @@ import im.vector.matrix.android.api.util.JsonDict
 import im.vector.matrix.android.api.util.Optional
 import im.vector.matrix.android.api.util.toOptional
 import im.vector.matrix.android.internal.crypto.model.CryptoDeviceInfo
+import im.vector.matrix.android.internal.crypto.model.rest.DeviceInfo
+import im.vector.matrix.android.internal.crypto.store.PrivateKeysInfo
 import im.vector.matrix.android.internal.session.sync.model.accountdata.UserAccountDataEvent
 import io.reactivex.Observable
 import io.reactivex.Single
@@ -58,6 +60,13 @@ class RxSession(private val session: Session) {
                 }
     }
 
+    fun liveMyDeviceInfo(): Observable<List<DeviceInfo>> {
+        return session.cryptoService().getLiveMyDevicesInfo().asObservable()
+                .startWithCallable {
+                    session.cryptoService().getMyDevicesInfo()
+                }
+    }
+
     fun liveSyncState(): Observable<SyncState> {
         return session.getSyncStateLive().asObservable()
     }
@@ -81,8 +90,8 @@ class RxSession(private val session: Session) {
         return session.getIgnoredUsersLive().asObservable()
     }
 
-    fun livePagedUsers(filter: String? = null): Observable<PagedList<User>> {
-        return session.getPagedUsersLive(filter).asObservable()
+    fun livePagedUsers(filter: String? = null, excludedUserIds: Set<String>? = null): Observable<PagedList<User>> {
+        return session.getPagedUsersLive(filter, excludedUserIds).asObservable()
     }
 
     fun createRoom(roomParams: CreateRoomParams): Single<String> = singleBuilder {
@@ -123,6 +132,13 @@ class RxSession(private val session: Session) {
                 }
     }
 
+    fun liveCrossSigningPrivateKeys(): Observable<Optional<PrivateKeysInfo>> {
+        return session.cryptoService().crossSigningService().getLiveCrossSigningPrivateKeys().asObservable()
+                .startWithCallable {
+                    session.cryptoService().crossSigningService().getCrossSigningPrivateKeys().toOptional()
+                }
+    }
+
     fun liveAccountData(types: Set<String>): Observable<List<UserAccountDataEvent>> {
         return session.getLiveAccountDataEvents(types).asObservable()
                 .startWithCallable {

matrix-sdk-android/build.gradle

@@ -71,6 +71,15 @@ android {
     kotlinOptions {
         jvmTarget = "1.8"
     }
+
+    sourceSets {
+        androidTest {
+            java.srcDirs += "src/sharedTest/java"
+        }
+        test {
+            java.srcDirs += "src/sharedTest/java"
+        }
+    }
 }
 
 static def gitRevision() {
@@ -160,6 +169,8 @@ dependencies {
     testImplementation 'io.mockk:mockk:1.9.2.kotlin12'
     testImplementation 'org.amshove.kluent:kluent-android:1.44'
     testImplementation "org.jetbrains.kotlinx:kotlinx-coroutines-android:$coroutines_version"
+    // Plant Timber tree for test
+    testImplementation 'net.lachlanmckee:timber-junit-rule:1.0.1'
 
     androidTestImplementation 'androidx.test:core:1.2.0'
     androidTestImplementation 'androidx.test:runner:1.2.0'
@@ -171,5 +182,6 @@ dependencies {
     androidTestImplementation 'io.mockk:mockk-android:1.9.2.kotlin12'
     androidTestImplementation "androidx.arch.core:core-testing:$arch_version"
     androidTestImplementation "org.jetbrains.kotlinx:kotlinx-coroutines-android:$coroutines_version"
-
+    // Plant Timber tree for test
+    androidTestImplementation 'net.lachlanmckee:timber-junit-rule:1.0.1'
 }

matrix-sdk-android/src/androidTest/java/im/vector/matrix/android/InstrumentedTest.kt

@@ -18,10 +18,15 @@ package im.vector.matrix.android
 
 import android.content.Context
 import androidx.test.core.app.ApplicationProvider
+import im.vector.matrix.android.test.shared.createTimberTestRule
+import org.junit.Rule
 import java.io.File
 
 interface InstrumentedTest {
 
+    @Rule
+    fun timberTestRule() = createTimberTestRule()
+
     fun context(): Context {
         return ApplicationProvider.getApplicationContext()
     }

matrix-sdk-android/src/androidTest/java/im/vector/matrix/android/account/ChangePasswordTest.kt

@@ -0,0 +1,60 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package im.vector.matrix.android.account
+
+import im.vector.matrix.android.InstrumentedTest
+import im.vector.matrix.android.api.failure.isInvalidPassword
+import im.vector.matrix.android.common.CommonTestHelper
+import im.vector.matrix.android.common.SessionTestParams
+import im.vector.matrix.android.common.TestConstants
+import org.amshove.kluent.shouldBeTrue
+import org.junit.FixMethodOrder
+import org.junit.Test
+import org.junit.runner.RunWith
+import org.junit.runners.JUnit4
+import org.junit.runners.MethodSorters
+
+@RunWith(JUnit4::class)
+@FixMethodOrder(MethodSorters.JVM)
+class ChangePasswordTest : InstrumentedTest {
+
+    private val commonTestHelper = CommonTestHelper(context())
+
+    companion object {
+        private const val NEW_PASSWORD = "this is a new password"
+    }
+
+    @Test
+    fun changePasswordTest() {
+        val session = commonTestHelper.createAccount(TestConstants.USER_ALICE, SessionTestParams(withInitialSync = false))
+
+        // Change password
+        commonTestHelper.doSync<Unit> {
+            session.changePassword(TestConstants.PASSWORD, NEW_PASSWORD, it)
+        }
+
+        // Try to login with the previous password, it will fail
+        val throwable = commonTestHelper.logAccountWithError(session.myUserId, TestConstants.PASSWORD)
+        throwable.isInvalidPassword().shouldBeTrue()
+
+        // Try to login with the new password, should work
+        val session2 = commonTestHelper.logIntoAccount(session.myUserId, NEW_PASSWORD, SessionTestParams(withInitialSync = false))
+
+        commonTestHelper.signOutAndClose(session)
+        commonTestHelper.signOutAndClose(session2)
+    }
+}

matrix-sdk-android/src/androidTest/java/im/vector/matrix/android/account/DeactivateAccountTest.kt

@@ -0,0 +1,88 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package im.vector.matrix.android.account
+
+import im.vector.matrix.android.InstrumentedTest
+import im.vector.matrix.android.api.auth.data.LoginFlowResult
+import im.vector.matrix.android.api.auth.registration.RegistrationResult
+import im.vector.matrix.android.api.failure.Failure
+import im.vector.matrix.android.api.failure.MatrixError
+import im.vector.matrix.android.common.CommonTestHelper
+import im.vector.matrix.android.common.SessionTestParams
+import im.vector.matrix.android.common.TestConstants
+import im.vector.matrix.android.common.TestMatrixCallback
+import org.junit.Assert.assertTrue
+import org.junit.FixMethodOrder
+import org.junit.Test
+import org.junit.runner.RunWith
+import org.junit.runners.JUnit4
+import org.junit.runners.MethodSorters
+
+@RunWith(JUnit4::class)
+@FixMethodOrder(MethodSorters.JVM)
+class DeactivateAccountTest : InstrumentedTest {
+
+    private val commonTestHelper = CommonTestHelper(context())
+
+    @Test
+    fun deactivateAccountTest() {
+        val session = commonTestHelper.createAccount(TestConstants.USER_ALICE, SessionTestParams(withInitialSync = false))
+
+        // Deactivate the account
+        commonTestHelper.doSync<Unit> {
+            session.deactivateAccount(TestConstants.PASSWORD, false, it)
+        }
+
+        // Try to login on the previous account, it will fail (M_USER_DEACTIVATED)
+        val throwable = commonTestHelper.logAccountWithError(session.myUserId, TestConstants.PASSWORD)
+
+        // Test the error
+        assertTrue(throwable is Failure.ServerError
+                && throwable.error.code == MatrixError.M_USER_DEACTIVATED
+                && throwable.error.message == "This account has been deactivated")
+
+        // Try to create an account with the deactivate account user id, it will fail (M_USER_IN_USE)
+        val hs = commonTestHelper.createHomeServerConfig()
+
+        commonTestHelper.doSync<LoginFlowResult> {
+            commonTestHelper.matrix.authenticationService.getLoginFlow(hs, it)
+        }
+
+        var accountCreationError: Throwable? = null
+        commonTestHelper.waitWithLatch {
+            commonTestHelper.matrix.authenticationService
+                    .getRegistrationWizard()
+                    .createAccount(session.myUserId.substringAfter("@").substringBefore(":"),
+                            TestConstants.PASSWORD,
+                            null,
+                            object : TestMatrixCallback<RegistrationResult>(it, false) {
+                                override fun onFailure(failure: Throwable) {
+                                    accountCreationError = failure
+                                    super.onFailure(failure)
+                                }
+                            })
+        }
+
+        // Test the error
+        accountCreationError.let {
+            assertTrue(it is Failure.ServerError
+                    && it.error.code == MatrixError.M_USER_IN_USE)
+        }
+
+        // No need to close the session, it has been deactivated
+    }
+}

matrix-sdk-android/src/androidTest/java/im/vector/matrix/android/common/CommonTestHelper.kt

@@ -183,9 +183,9 @@ class CommonTestHelper(context: Context) {
      * @param testParams test params about the session
      * @return the session associated with the existing account
      */
-    private fun logIntoAccount(userId: String,
-                               password: String,
-                               testParams: SessionTestParams): Session {
+    fun logIntoAccount(userId: String,
+                       password: String,
+                       testParams: SessionTestParams): Session {
         val session = logAccountAndSync(userId, password, testParams)
         assertNotNull(session)
         return session
@@ -260,14 +260,45 @@ class CommonTestHelper(context: Context) {
         return session
     }
 
+    /**
+     * Log into the account and expect an error
+     *
+     * @param userName the account username
+     * @param password the password
+     */
+    fun logAccountWithError(userName: String,
+                            password: String): Throwable {
+        val hs = createHomeServerConfig()
+
+        doSync<LoginFlowResult> {
+            matrix.authenticationService
+                    .getLoginFlow(hs, it)
+        }
+
+        var requestFailure: Throwable? = null
+        waitWithLatch { latch ->
+            matrix.authenticationService
+                    .getLoginWizard()
+                    .login(userName, password, "myDevice", object : TestMatrixCallback<Session>(latch, onlySuccessful = false) {
+                        override fun onFailure(failure: Throwable) {
+                            requestFailure = failure
+                            super.onFailure(failure)
+                        }
+                    })
+        }
+
+        assertNotNull(requestFailure)
+        return requestFailure!!
+    }
+
     /**
      * Await for a latch and ensure the result is true
      *
      * @param latch
      * @throws InterruptedException
      */
-    fun await(latch: CountDownLatch, timout: Long? = TestConstants.timeOutMillis) {
-        assertTrue(latch.await(timout ?: TestConstants.timeOutMillis, TimeUnit.MILLISECONDS))
+    fun await(latch: CountDownLatch, timeout: Long? = TestConstants.timeOutMillis) {
+        assertTrue(latch.await(timeout ?: TestConstants.timeOutMillis, TimeUnit.MILLISECONDS))
     }
 
     fun retryPeriodicallyWithLatch(latch: CountDownLatch, condition: (() -> Boolean)) {
@@ -282,10 +313,10 @@ class CommonTestHelper(context: Context) {
         }
     }
 
-    fun waitWithLatch(timout: Long? = TestConstants.timeOutMillis, block: (CountDownLatch) -> Unit) {
+    fun waitWithLatch(timeout: Long? = TestConstants.timeOutMillis, block: (CountDownLatch) -> Unit) {
         val latch = CountDownLatch(1)
         block(latch)
-        await(latch, timout)
+        await(latch, timeout)
     }
 
     // Transform a method with a MatrixCallback to a synchronous method

matrix-sdk-android/src/androidTest/java/im/vector/matrix/android/common/CryptoTestHelper.kt

@@ -22,6 +22,7 @@ import im.vector.matrix.android.api.session.Session
 import im.vector.matrix.android.api.session.events.model.Event
 import im.vector.matrix.android.api.session.events.model.EventType
 import im.vector.matrix.android.api.session.events.model.toContent
+import im.vector.matrix.android.api.session.room.Room
 import im.vector.matrix.android.api.session.room.model.Membership
 import im.vector.matrix.android.api.session.room.model.RoomSummary
 import im.vector.matrix.android.api.session.room.model.create.CreateRoomParams
@@ -40,8 +41,6 @@ import kotlinx.coroutines.runBlocking
 import org.junit.Assert.assertEquals
 import org.junit.Assert.assertNotNull
 import org.junit.Assert.assertNull
-import org.junit.Assert.assertTrue
-import java.util.HashMap
 import java.util.concurrent.CountDownLatch
 
 class CryptoTestHelper(private val mTestHelper: CommonTestHelper) {
@@ -140,64 +139,38 @@ class CryptoTestHelper(private val mTestHelper: CommonTestHelper) {
      * @return Alice, Bob and Sam session
      */
     fun doE2ETestWithAliceAndBobAndSamInARoom(): CryptoTestData {
-        val statuses = HashMap<String, String>()
-
         val cryptoTestData = doE2ETestWithAliceAndBobInARoom()
         val aliceSession = cryptoTestData.firstSession
         val aliceRoomId = cryptoTestData.roomId
 
         val room = aliceSession.getRoom(aliceRoomId)!!
 
-        val samSession = mTestHelper.createAccount(TestConstants.USER_SAM, defaultSessionParams)
-
-        val lock1 = CountDownLatch(2)
-
-//        val samEventListener = object : MXEventListener() {
-//            override fun onNewRoom(roomId: String) {
-//                if (TextUtils.equals(roomId, aliceRoomId)) {
-//                    if (!statuses.containsKey("onNewRoom")) {
-//                        statuses["onNewRoom"] = "onNewRoom"
-//                        lock1.countDown()
-//                    }
-//                }
-//            }
-//        }
-//
-//        samSession.dataHandler.addListener(samEventListener)
-
-        room.invite(samSession.myUserId, null, object : TestMatrixCallback<Unit>(lock1) {
-            override fun onSuccess(data: Unit) {
-                statuses["invite"] = "invite"
-                super.onSuccess(data)
-            }
-        })
-
-        mTestHelper.await(lock1)
-
-        assertTrue(statuses.containsKey("invite") && statuses.containsKey("onNewRoom"))
-
-//        samSession.dataHandler.removeListener(samEventListener)
-
-        val lock2 = CountDownLatch(1)
-
-        samSession.joinRoom(aliceRoomId, null, object : TestMatrixCallback<Unit>(lock2) {
-            override fun onSuccess(data: Unit) {
-                statuses["joinRoom"] = "joinRoom"
-                super.onSuccess(data)
-            }
-        })
-
-        mTestHelper.await(lock2)
-        assertTrue(statuses.containsKey("joinRoom"))
+        val samSession = createSamAccountAndInviteToTheRoom(room)
 
         // wait the initial sync
         SystemClock.sleep(1000)
 
-//        samSession.dataHandler.removeListener(samEventListener)
-
         return CryptoTestData(aliceSession, aliceRoomId, cryptoTestData.secondSession, samSession)
     }
 
+    /**
+     * Create Sam account and invite him in the room. He will accept the invitation
+     * @Return Sam session
+     */
+    fun createSamAccountAndInviteToTheRoom(room: Room): Session {
+        val samSession = mTestHelper.createAccount(TestConstants.USER_SAM, defaultSessionParams)
+
+        mTestHelper.doSync<Unit> {
+            room.invite(samSession.myUserId, null, it)
+        }
+
+        mTestHelper.doSync<Unit> {
+            samSession.joinRoom(room.roomId, null, it)
+        }
+
+        return samSession
+    }
+
     /**
      * @return Alice and Bob sessions
      */

matrix-sdk-android/src/androidTest/java/im/vector/matrix/android/internal/crypto/CryptoStoreHelper.kt

@@ -20,6 +20,8 @@ import im.vector.matrix.android.api.auth.data.Credentials
 import im.vector.matrix.android.internal.crypto.store.IMXCryptoStore
 import im.vector.matrix.android.internal.crypto.store.db.RealmCryptoStore
 import im.vector.matrix.android.internal.crypto.store.db.RealmCryptoStoreModule
+import im.vector.matrix.android.internal.crypto.store.db.mapper.CrossSigningKeysMapper
+import im.vector.matrix.android.internal.di.MoshiProvider
 import io.realm.RealmConfiguration
 import kotlin.random.Random
 
@@ -31,6 +33,7 @@ internal class CryptoStoreHelper {
                         .name("test.realm")
                         .modules(RealmCryptoStoreModule())
                         .build(),
+                crossSigningKeysMapper = CrossSigningKeysMapper(MoshiProvider.providesMoshi()),
                 credentials = createCredential())
     }
 

matrix-sdk-android/src/androidTest/java/im/vector/matrix/android/internal/crypto/UnwedgingTest.kt

@@ -0,0 +1,247 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package im.vector.matrix.android.internal.crypto
+
+import androidx.test.ext.junit.runners.AndroidJUnit4
+import im.vector.matrix.android.InstrumentedTest
+import im.vector.matrix.android.api.extensions.tryThis
+import im.vector.matrix.android.api.session.crypto.MXCryptoError
+import im.vector.matrix.android.api.session.events.model.EventType
+import im.vector.matrix.android.api.session.events.model.toModel
+import im.vector.matrix.android.api.session.room.timeline.Timeline
+import im.vector.matrix.android.api.session.room.timeline.TimelineEvent
+import im.vector.matrix.android.api.session.room.timeline.TimelineSettings
+import im.vector.matrix.android.common.CommonTestHelper
+import im.vector.matrix.android.common.CryptoTestHelper
+import im.vector.matrix.android.common.TestConstants
+import im.vector.matrix.android.internal.crypto.model.OlmSessionWrapper
+import im.vector.matrix.android.internal.crypto.model.event.EncryptedEventContent
+import im.vector.matrix.android.internal.crypto.model.rest.UserPasswordAuth
+import im.vector.matrix.android.internal.crypto.store.db.deserializeFromRealm
+import im.vector.matrix.android.internal.crypto.store.db.serializeForRealm
+import org.amshove.kluent.shouldBe
+import org.junit.Assert
+import org.junit.Before
+import org.junit.FixMethodOrder
+import org.junit.Test
+import org.junit.runner.RunWith
+import org.junit.runners.MethodSorters
+import org.matrix.olm.OlmSession
+import timber.log.Timber
+import java.util.concurrent.CountDownLatch
+
+/**
+ * Ref:
+ * - https://github.com/matrix-org/matrix-doc/pull/1719
+ * - https://matrix.org/docs/spec/client_server/latest#recovering-from-undecryptable-messages
+ * - https://github.com/matrix-org/matrix-js-sdk/pull/780
+ * - https://github.com/matrix-org/matrix-ios-sdk/pull/778
+ * - https://github.com/matrix-org/matrix-ios-sdk/pull/784
+ */
+@RunWith(AndroidJUnit4::class)
+@FixMethodOrder(MethodSorters.JVM)
+class UnwedgingTest : InstrumentedTest {
+
+    private lateinit var messagesReceivedByBob: List<TimelineEvent>
+    private val mTestHelper = CommonTestHelper(context())
+    private val mCryptoTestHelper = CryptoTestHelper(mTestHelper)
+
+    @Before
+    fun init() {
+        messagesReceivedByBob = emptyList()
+    }
+
+    /**
+     * - Alice & Bob in a e2e room
+     * - Alice sends a 1st message with a 1st megolm session
+     * - Store the olm session between A&B devices
+     * - Alice sends a 2nd message with a 2nd megolm session
+     * - Simulate Alice using a backup of her OS and make her crypto state like after the first message
+     * - Alice sends a 3rd message with a 3rd megolm session but a wedged olm session
+     *
+     * What Bob must see:
+     * -> No issue with the 2 first messages
+     * -> The third event must fail to decrypt at first because Bob the olm session is wedged
+     * -> This is automatically fixed after SDKs restarted the olm session
+     */
+    @Test
+    fun testUnwedging() {
+        val cryptoTestData = mCryptoTestHelper.doE2ETestWithAliceAndBobInARoom()
+
+        val aliceSession = cryptoTestData.firstSession
+        val aliceRoomId = cryptoTestData.roomId
+        val bobSession = cryptoTestData.secondSession!!
+
+        val aliceCryptoStore = (aliceSession.cryptoService() as DefaultCryptoService).cryptoStoreForTesting
+
+        // bobSession.cryptoService().setWarnOnUnknownDevices(false)
+        // aliceSession.cryptoService().setWarnOnUnknownDevices(false)
+
+        val roomFromBobPOV = bobSession.getRoom(aliceRoomId)!!
+        val roomFromAlicePOV = aliceSession.getRoom(aliceRoomId)!!
+
+        val bobTimeline = roomFromBobPOV.createTimeline(null, TimelineSettings(20))
+        bobTimeline.start()
+
+        val bobFinalLatch = CountDownLatch(1)
+        val bobHasThreeDecryptedEventsListener = object : Timeline.Listener {
+            override fun onTimelineFailure(throwable: Throwable) {
+                // noop
+            }
+
+            override fun onNewTimelineEvents(eventIds: List<String>) {
+                // noop
+            }
+
+            override fun onTimelineUpdated(snapshot: List<TimelineEvent>) {
+                val decryptedEventReceivedByBob = snapshot.filter { it.root.type == EventType.ENCRYPTED }
+                Timber.d("Bob can now decrypt ${decryptedEventReceivedByBob.size} messages")
+                if (decryptedEventReceivedByBob.size == 3) {
+                    if (decryptedEventReceivedByBob[0].root.mCryptoError == MXCryptoError.ErrorType.UNKNOWN_INBOUND_SESSION_ID) {
+                        bobFinalLatch.countDown()
+                    }
+                }
+            }
+        }
+        bobTimeline.addListener(bobHasThreeDecryptedEventsListener)
+
+        var latch = CountDownLatch(1)
+        var bobEventsListener = createEventListener(latch, 1)
+        bobTimeline.addListener(bobEventsListener)
+        messagesReceivedByBob = emptyList()
+
+        // - Alice sends a 1st message with a 1st megolm session
+        roomFromAlicePOV.sendTextMessage("First message")
+
+        // Wait for the message to be received by Bob
+        mTestHelper.await(latch)
+        bobTimeline.removeListener(bobEventsListener)
+
+        messagesReceivedByBob.size shouldBe 1
+        val firstMessageSession = messagesReceivedByBob[0].root.content.toModel<EncryptedEventContent>()!!.sessionId!!
+
+        //  - Store the olm session between A&B devices
+        // Let us pickle our session with bob here so we can later unpickle it
+        // and wedge our session.
+        val sessionIdsForBob = aliceCryptoStore.getDeviceSessionIds(bobSession.cryptoService().getMyDevice().identityKey()!!)
+        sessionIdsForBob!!.size shouldBe 1
+        val olmSession = aliceCryptoStore.getDeviceSession(sessionIdsForBob.first(), bobSession.cryptoService().getMyDevice().identityKey()!!)!!
+
+        val oldSession = serializeForRealm(olmSession.olmSession)
+
+        aliceSession.cryptoService().discardOutboundSession(roomFromAlicePOV.roomId)
+        Thread.sleep(6_000)
+
+        latch = CountDownLatch(1)
+        bobEventsListener = createEventListener(latch, 2)
+        bobTimeline.addListener(bobEventsListener)
+        messagesReceivedByBob = emptyList()
+
+        Timber.i("## CRYPTO | testUnwedging:  Alice sends a 2nd message with a 2nd megolm session")
+        // - Alice sends a 2nd message with a 2nd megolm session
+        roomFromAlicePOV.sendTextMessage("Second message")
+
+        // Wait for the message to be received by Bob
+        mTestHelper.await(latch)
+        bobTimeline.removeListener(bobEventsListener)
+
+        messagesReceivedByBob.size shouldBe 2
+        // Session should have changed
+        val secondMessageSession = messagesReceivedByBob[0].root.content.toModel<EncryptedEventContent>()!!.sessionId!!
+        Assert.assertNotEquals(firstMessageSession, secondMessageSession)
+
+        // Let us wedge the session now. Set crypto state like after the first message
+        Timber.i("## CRYPTO | testUnwedging: wedge the session now. Set crypto state like after the first message")
+
+        aliceCryptoStore.storeSession(OlmSessionWrapper(deserializeFromRealm<OlmSession>(oldSession)!!), bobSession.cryptoService().getMyDevice().identityKey()!!)
+        Thread.sleep(6_000)
+
+        // Force new session, and key share
+        aliceSession.cryptoService().discardOutboundSession(roomFromAlicePOV.roomId)
+
+        // Wait for the message to be received by Bob
+        mTestHelper.waitWithLatch {
+            bobEventsListener = createEventListener(it, 3)
+            bobTimeline.addListener(bobEventsListener)
+            messagesReceivedByBob = emptyList()
+
+            Timber.i("## CRYPTO | testUnwedging: Alice sends a 3rd message with a 3rd megolm session but a wedged olm session")
+            // - Alice sends a 3rd message with a 3rd megolm session but a wedged olm session
+            roomFromAlicePOV.sendTextMessage("Third message")
+            // Bob should not be able to decrypt, because the session key could not be sent
+        }
+        bobTimeline.removeListener(bobEventsListener)
+
+        messagesReceivedByBob.size shouldBe 3
+
+        val thirdMessageSession = messagesReceivedByBob[0].root.content.toModel<EncryptedEventContent>()!!.sessionId!!
+        Timber.i("## CRYPTO | testUnwedging: third message session ID $thirdMessageSession")
+        Assert.assertNotEquals(secondMessageSession, thirdMessageSession)
+
+        Assert.assertEquals(EventType.ENCRYPTED, messagesReceivedByBob[0].root.getClearType())
+        Assert.assertEquals(EventType.MESSAGE, messagesReceivedByBob[1].root.getClearType())
+        Assert.assertEquals(EventType.MESSAGE, messagesReceivedByBob[2].root.getClearType())
+        // Bob Should not be able to decrypt last message, because session could not be sent as the olm channel was wedged
+        mTestHelper.await(bobFinalLatch)
+        bobTimeline.removeListener(bobHasThreeDecryptedEventsListener)
+
+        // It's a trick to force key request on fail to decrypt
+        mTestHelper.doSync<Unit> {
+            bobSession.cryptoService().crossSigningService()
+                    .initializeCrossSigning(UserPasswordAuth(
+                            user = bobSession.myUserId,
+                            password = TestConstants.PASSWORD
+                    ), it)
+        }
+
+        // Wait until we received back the key
+        mTestHelper.waitWithLatch {
+            mTestHelper.retryPeriodicallyWithLatch(it) {
+                // we should get back the key and be able to decrypt
+                val result = tryThis {
+                    bobSession.cryptoService().decryptEvent(messagesReceivedByBob[0].root, "")
+                }
+                Timber.i("## CRYPTO | testUnwedging: decrypt result  ${result?.clearEvent}")
+                result != null
+            }
+        }
+
+        bobTimeline.dispose()
+
+        cryptoTestData.cleanUp(mTestHelper)
+    }
+
+    private fun createEventListener(latch: CountDownLatch, expectedNumberOfMessages: Int): Timeline.Listener {
+        return object : Timeline.Listener {
+            override fun onTimelineFailure(throwable: Throwable) {
+                // noop
+            }
+
+            override fun onNewTimelineEvents(eventIds: List<String>) {
+                // noop
+            }
+
+            override fun onTimelineUpdated(snapshot: List<TimelineEvent>) {
+                messagesReceivedByBob = snapshot.filter { it.root.type == EventType.ENCRYPTED }
+
+                if (messagesReceivedByBob.size == expectedNumberOfMessages) {
+                    latch.countDown()
+                }
+            }
+        }
+    }
+}

matrix-sdk-android/src/androidTest/java/im/vector/matrix/android/internal/crypto/keysbackup/KeysBackupScenarioData.kt

@@ -0,0 +1,35 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package im.vector.matrix.android.internal.crypto.keysbackup
+
+import im.vector.matrix.android.api.session.Session
+import im.vector.matrix.android.common.CommonTestHelper
+import im.vector.matrix.android.common.CryptoTestData
+import im.vector.matrix.android.internal.crypto.model.OlmInboundGroupSessionWrapper2
+
+/**
+ * Data class to store result of [KeysBackupTestHelper.createKeysBackupScenarioWithPassword]
+ */
+data class KeysBackupScenarioData(val cryptoTestData: CryptoTestData,
+                                  val aliceKeys: List<OlmInboundGroupSessionWrapper2>,
+                                  val prepareKeysBackupDataResult: PrepareKeysBackupDataResult,
+                                  val aliceSession2: Session) {
+    fun cleanUp(testHelper: CommonTestHelper) {
+        cryptoTestData.cleanUp(testHelper)
+        testHelper.signOutAndClose(aliceSession2)
+    }
+}

matrix-sdk-android/src/androidTest/java/im/vector/matrix/android/internal/crypto/keysbackup/KeysBackupTest.kt

@@ -20,27 +20,19 @@ import androidx.test.ext.junit.runners.AndroidJUnit4
 import im.vector.matrix.android.InstrumentedTest
 import im.vector.matrix.android.api.listeners.ProgressListener
 import im.vector.matrix.android.api.listeners.StepProgressListener
-import im.vector.matrix.android.api.session.Session
-import im.vector.matrix.android.api.session.crypto.keysbackup.KeysBackupService
 import im.vector.matrix.android.api.session.crypto.keysbackup.KeysBackupState
 import im.vector.matrix.android.api.session.crypto.keysbackup.KeysBackupStateListener
 import im.vector.matrix.android.common.CommonTestHelper
-import im.vector.matrix.android.common.CryptoTestData
 import im.vector.matrix.android.common.CryptoTestHelper
-import im.vector.matrix.android.common.SessionTestParams
 import im.vector.matrix.android.common.TestConstants
 import im.vector.matrix.android.common.TestMatrixCallback
-import im.vector.matrix.android.common.assertDictEquals
-import im.vector.matrix.android.common.assertListEquals
 import im.vector.matrix.android.internal.crypto.MXCRYPTO_ALGORITHM_MEGOLM_BACKUP
-import im.vector.matrix.android.internal.crypto.MegolmSessionData
 import im.vector.matrix.android.internal.crypto.crosssigning.DeviceTrustLevel
 import im.vector.matrix.android.internal.crypto.keysbackup.model.KeysBackupVersionTrust
 import im.vector.matrix.android.internal.crypto.keysbackup.model.MegolmBackupCreationInfo
 import im.vector.matrix.android.internal.crypto.keysbackup.model.rest.KeysVersion
 import im.vector.matrix.android.internal.crypto.keysbackup.model.rest.KeysVersionResult
 import im.vector.matrix.android.internal.crypto.model.ImportRoomKeysResult
-import im.vector.matrix.android.internal.crypto.model.OlmInboundGroupSessionWrapper
 import org.junit.Assert.assertEquals
 import org.junit.Assert.assertFalse
 import org.junit.Assert.assertNotNull
@@ -61,9 +53,7 @@ class KeysBackupTest : InstrumentedTest {
 
     private val mTestHelper = CommonTestHelper(context())
     private val mCryptoTestHelper = CryptoTestHelper(mTestHelper)
-
-    private val defaultSessionParams = SessionTestParams(withInitialSync = false)
-    private val defaultSessionParamsWithInitialSync = SessionTestParams(withInitialSync = true)
+    private val mKeysBackupTestHelper = KeysBackupTestHelper(mTestHelper, mCryptoTestHelper)
 
     /**
      * - From doE2ETestWithAliceAndBobInARoomWithEncryptedMessages, we should have no backed up keys
@@ -110,7 +100,7 @@ class KeysBackupTest : InstrumentedTest {
      */
     @Test
     fun prepareKeysBackupVersionTest() {
-        val bobSession = mTestHelper.createAccount(TestConstants.USER_BOB, defaultSessionParams)
+        val bobSession = mTestHelper.createAccount(TestConstants.USER_BOB, KeysBackupTestConstants.defaultSessionParams)
 
         assertNotNull(bobSession.cryptoService().keysBackupService())
 
@@ -139,7 +129,7 @@ class KeysBackupTest : InstrumentedTest {
      */
     @Test
     fun createKeysBackupVersionTest() {
-        val bobSession = mTestHelper.createAccount(TestConstants.USER_BOB, defaultSessionParams)
+        val bobSession = mTestHelper.createAccount(TestConstants.USER_BOB, KeysBackupTestConstants.defaultSessionParams)
 
         val keysBackup = bobSession.cryptoService().keysBackupService()
 
@@ -182,7 +172,7 @@ class KeysBackupTest : InstrumentedTest {
 
         val stateObserver = StateObserver(keysBackup, latch, 5)
 
-        prepareAndCreateKeysBackupData(keysBackup)
+        mKeysBackupTestHelper.prepareAndCreateKeysBackupData(keysBackup)
 
         mTestHelper.await(latch)
 
@@ -216,7 +206,7 @@ class KeysBackupTest : InstrumentedTest {
 
         val stateObserver = StateObserver(keysBackup)
 
-        prepareAndCreateKeysBackupData(keysBackup)
+        mKeysBackupTestHelper.prepareAndCreateKeysBackupData(keysBackup)
 
         // Check that backupAllGroupSessions returns valid data
         val nbOfKeys = cryptoTestData.firstSession.cryptoService().inboundGroupSessionsCount(false)
@@ -263,7 +253,7 @@ class KeysBackupTest : InstrumentedTest {
         // - Pick a megolm key
         val session = keysBackup.store.inboundGroupSessionsToBackup(1)[0]
 
-        val keyBackupCreationInfo = prepareAndCreateKeysBackupData(keysBackup).megolmBackupCreationInfo
+        val keyBackupCreationInfo = mKeysBackupTestHelper.prepareAndCreateKeysBackupData(keysBackup).megolmBackupCreationInfo
 
         // - Check encryptGroupSession() returns stg
         val keyBackupData = keysBackup.encryptGroupSession(session)
@@ -281,7 +271,7 @@ class KeysBackupTest : InstrumentedTest {
                         decryption!!)
         assertNotNull(sessionData)
         // - Compare the decrypted megolm key with the original one
-        assertKeysEquals(session.exportKeys(), sessionData)
+        mKeysBackupTestHelper.assertKeysEquals(session.exportKeys(), sessionData)
 
         stateObserver.stopAndCheckStates(null)
         cryptoTestData.cleanUp(mTestHelper)
@@ -295,7 +285,7 @@ class KeysBackupTest : InstrumentedTest {
      */
     @Test
     fun restoreKeysBackupTest() {
-        val testData = createKeysBackupScenarioWithPassword(null)
+        val testData = mKeysBackupTestHelper.createKeysBackupScenarioWithPassword(null)
 
         // - Restore the e2e backup from the homeserver
         val importRoomKeysResult = mTestHelper.doSync<ImportRoomKeysResult> {
@@ -308,7 +298,7 @@ class KeysBackupTest : InstrumentedTest {
             )
         }
 
-        checkRestoreSuccess(testData, importRoomKeysResult.totalNumberOfKeys, importRoomKeysResult.successfullyNumberOfImportedKeys)
+        mKeysBackupTestHelper.checkRestoreSuccess(testData, importRoomKeysResult.totalNumberOfKeys, importRoomKeysResult.successfullyNumberOfImportedKeys)
 
         testData.cleanUp(mTestHelper)
     }
@@ -329,7 +319,7 @@ class KeysBackupTest : InstrumentedTest {
 //    fun restoreKeysBackupAndKeyShareRequestTest() {
 //        fail("Check with Valere for this test. I think we do not send key share request")
 //
-//        val testData = createKeysBackupScenarioWithPassword(null)
+//        val testData = mKeysBackupTestHelper.createKeysBackupScenarioWithPassword(null)
 //
 //        // - Check the SDK sent key share requests
 //        val cryptoStore2 = (testData.aliceSession2.cryptoService().keysBackupService() as DefaultKeysBackupService).store
@@ -352,7 +342,7 @@ class KeysBackupTest : InstrumentedTest {
 //            )
 //        }
 //
-//        checkRestoreSuccess(testData, importRoomKeysResult.totalNumberOfKeys, importRoomKeysResult.successfullyNumberOfImportedKeys)
+//        mKeysBackupTestHelper.checkRestoreSuccess(testData, importRoomKeysResult.totalNumberOfKeys, importRoomKeysResult.successfullyNumberOfImportedKeys)
 //
 //        // - There must be no more pending key share requests
 //        val unsentRequestAfterRestoration = cryptoStore2
@@ -380,7 +370,7 @@ class KeysBackupTest : InstrumentedTest {
     fun trustKeyBackupVersionTest() {
         // - Do an e2e backup to the homeserver with a recovery key
         // - And log Alice on a new device
-        val testData = createKeysBackupScenarioWithPassword(null)
+        val testData = mKeysBackupTestHelper.createKeysBackupScenarioWithPassword(null)
 
         val stateObserver = StateObserver(testData.aliceSession2.cryptoService().keysBackupService())
 
@@ -399,7 +389,7 @@ class KeysBackupTest : InstrumentedTest {
         }
 
         // Wait for backup state to be ReadyToBackUp
-        waitForKeysBackupToBeInState(testData.aliceSession2, KeysBackupState.ReadyToBackUp)
+        mKeysBackupTestHelper.waitForKeysBackupToBeInState(testData.aliceSession2, KeysBackupState.ReadyToBackUp)
 
         // - Backup must be enabled on the new device, on the same version
         assertEquals(testData.prepareKeysBackupDataResult.version, testData.aliceSession2.cryptoService().keysBackupService().keysBackupVersion?.version)
@@ -439,7 +429,7 @@ class KeysBackupTest : InstrumentedTest {
     fun trustKeyBackupVersionWithRecoveryKeyTest() {
         // - Do an e2e backup to the homeserver with a recovery key
         // - And log Alice on a new device
-        val testData = createKeysBackupScenarioWithPassword(null)
+        val testData = mKeysBackupTestHelper.createKeysBackupScenarioWithPassword(null)
 
         val stateObserver = StateObserver(testData.aliceSession2.cryptoService().keysBackupService())
 
@@ -458,7 +448,7 @@ class KeysBackupTest : InstrumentedTest {
         }
 
         // Wait for backup state to be ReadyToBackUp
-        waitForKeysBackupToBeInState(testData.aliceSession2, KeysBackupState.ReadyToBackUp)
+        mKeysBackupTestHelper.waitForKeysBackupToBeInState(testData.aliceSession2, KeysBackupState.ReadyToBackUp)
 
         // - Backup must be enabled on the new device, on the same version
         assertEquals(testData.prepareKeysBackupDataResult.version, testData.aliceSession2.cryptoService().keysBackupService().keysBackupVersion?.version)
@@ -496,7 +486,7 @@ class KeysBackupTest : InstrumentedTest {
     fun trustKeyBackupVersionWithWrongRecoveryKeyTest() {
         // - Do an e2e backup to the homeserver with a recovery key
         // - And log Alice on a new device
-        val testData = createKeysBackupScenarioWithPassword(null)
+        val testData = mKeysBackupTestHelper.createKeysBackupScenarioWithPassword(null)
 
         val stateObserver = StateObserver(testData.aliceSession2.cryptoService().keysBackupService())
 
@@ -539,7 +529,7 @@ class KeysBackupTest : InstrumentedTest {
 
         // - Do an e2e backup to the homeserver with a password
         // - And log Alice on a new device
-        val testData = createKeysBackupScenarioWithPassword(password)
+        val testData = mKeysBackupTestHelper.createKeysBackupScenarioWithPassword(password)
 
         val stateObserver = StateObserver(testData.aliceSession2.cryptoService().keysBackupService())
 
@@ -558,7 +548,7 @@ class KeysBackupTest : InstrumentedTest {
         }
 
         // Wait for backup state to be ReadyToBackUp
-        waitForKeysBackupToBeInState(testData.aliceSession2, KeysBackupState.ReadyToBackUp)
+        mKeysBackupTestHelper.waitForKeysBackupToBeInState(testData.aliceSession2, KeysBackupState.ReadyToBackUp)
 
         // - Backup must be enabled on the new device, on the same version
         assertEquals(testData.prepareKeysBackupDataResult.version, testData.aliceSession2.cryptoService().keysBackupService().keysBackupVersion?.version)
@@ -599,7 +589,7 @@ class KeysBackupTest : InstrumentedTest {
 
         // - Do an e2e backup to the homeserver with a password
         // - And log Alice on a new device
-        val testData = createKeysBackupScenarioWithPassword(password)
+        val testData = mKeysBackupTestHelper.createKeysBackupScenarioWithPassword(password)
 
         val stateObserver = StateObserver(testData.aliceSession2.cryptoService().keysBackupService())
 
@@ -634,7 +624,7 @@ class KeysBackupTest : InstrumentedTest {
      */
     @Test
     fun restoreKeysBackupWithAWrongRecoveryKeyTest() {
-        val testData = createKeysBackupScenarioWithPassword(null)
+        val testData = mKeysBackupTestHelper.createKeysBackupScenarioWithPassword(null)
 
         // - Try to restore the e2e backup with a wrong recovery key
         val latch2 = CountDownLatch(1)
@@ -669,7 +659,7 @@ class KeysBackupTest : InstrumentedTest {
     fun testBackupWithPassword() {
         val password = "password"
 
-        val testData = createKeysBackupScenarioWithPassword(password)
+        val testData = mKeysBackupTestHelper.createKeysBackupScenarioWithPassword(password)
 
         // - Restore the e2e backup with the password
         val steps = ArrayList<StepProgressListener.Step>()
@@ -709,7 +699,7 @@ class KeysBackupTest : InstrumentedTest {
         assertEquals(50, (steps[103] as StepProgressListener.Step.ImportingKey).progress)
         assertEquals(100, (steps[104] as StepProgressListener.Step.ImportingKey).progress)
 
-        checkRestoreSuccess(testData, importRoomKeysResult.totalNumberOfKeys, importRoomKeysResult.successfullyNumberOfImportedKeys)
+        mKeysBackupTestHelper.checkRestoreSuccess(testData, importRoomKeysResult.totalNumberOfKeys, importRoomKeysResult.successfullyNumberOfImportedKeys)
 
         testData.cleanUp(mTestHelper)
     }
@@ -725,7 +715,7 @@ class KeysBackupTest : InstrumentedTest {
         val password = "password"
         val wrongPassword = "passw0rd"
 
-        val testData = createKeysBackupScenarioWithPassword(password)
+        val testData = mKeysBackupTestHelper.createKeysBackupScenarioWithPassword(password)
 
         // - Try to restore the e2e backup with a wrong password
         val latch2 = CountDownLatch(1)
@@ -760,7 +750,7 @@ class KeysBackupTest : InstrumentedTest {
     fun testUseRecoveryKeyToRestoreAPasswordBasedKeysBackup() {
         val password = "password"
 
-        val testData = createKeysBackupScenarioWithPassword(password)
+        val testData = mKeysBackupTestHelper.createKeysBackupScenarioWithPassword(password)
 
         // - Restore the e2e backup with the recovery key.
         val importRoomKeysResult = mTestHelper.doSync<ImportRoomKeysResult> {
@@ -773,7 +763,7 @@ class KeysBackupTest : InstrumentedTest {
             )
         }
 
-        checkRestoreSuccess(testData, importRoomKeysResult.totalNumberOfKeys, importRoomKeysResult.successfullyNumberOfImportedKeys)
+        mKeysBackupTestHelper.checkRestoreSuccess(testData, importRoomKeysResult.totalNumberOfKeys, importRoomKeysResult.successfullyNumberOfImportedKeys)
 
         testData.cleanUp(mTestHelper)
     }
@@ -786,7 +776,7 @@ class KeysBackupTest : InstrumentedTest {
      */
     @Test
     fun testUsePasswordToRestoreARecoveryKeyBasedKeysBackup() {
-        val testData = createKeysBackupScenarioWithPassword(null)
+        val testData = mKeysBackupTestHelper.createKeysBackupScenarioWithPassword(null)
 
         // - Try to restore the e2e backup with a password
         val latch2 = CountDownLatch(1)
@@ -825,7 +815,7 @@ class KeysBackupTest : InstrumentedTest {
         val stateObserver = StateObserver(keysBackup)
 
         // - Do an e2e backup to the homeserver
-        prepareAndCreateKeysBackupData(keysBackup)
+        mKeysBackupTestHelper.prepareAndCreateKeysBackupData(keysBackup)
 
         // Get key backup version from the home server
         val keysVersionResult = mTestHelper.doSync<KeysVersionResult?> {
@@ -870,13 +860,13 @@ class KeysBackupTest : InstrumentedTest {
 
         assertFalse(keysBackup.isEnabled)
 
-        val keyBackupCreationInfo = prepareAndCreateKeysBackupData(keysBackup)
+        val keyBackupCreationInfo = mKeysBackupTestHelper.prepareAndCreateKeysBackupData(keysBackup)
 
         assertTrue(keysBackup.isEnabled)
 
         // - Restart alice session
         // - Log Alice on a new device
-        val aliceSession2 = mTestHelper.logIntoAccount(cryptoTestData.firstSession.myUserId, defaultSessionParamsWithInitialSync)
+        val aliceSession2 = mTestHelper.logIntoAccount(cryptoTestData.firstSession.myUserId, KeysBackupTestConstants.defaultSessionParamsWithInitialSync)
 
         cryptoTestData.cleanUp(mTestHelper)
 
@@ -950,7 +940,7 @@ class KeysBackupTest : InstrumentedTest {
         })
 
         // - Make alice back up her keys to her homeserver
-        prepareAndCreateKeysBackupData(keysBackup)
+        mKeysBackupTestHelper.prepareAndCreateKeysBackupData(keysBackup)
 
         assertTrue(keysBackup.isEnabled)
 
@@ -1000,7 +990,7 @@ class KeysBackupTest : InstrumentedTest {
         val stateObserver = StateObserver(keysBackup)
 
         // - Make alice back up her keys to her homeserver
-        prepareAndCreateKeysBackupData(keysBackup)
+        mKeysBackupTestHelper.prepareAndCreateKeysBackupData(keysBackup)
 
         // Wait for keys backup to finish by asking again to backup keys.
         mTestHelper.doSync<Unit> {
@@ -1012,7 +1002,7 @@ class KeysBackupTest : InstrumentedTest {
         val aliceUserId = cryptoTestData.firstSession.myUserId
 
         // - Log Alice on a new device
-        val aliceSession2 = mTestHelper.logIntoAccount(aliceUserId, defaultSessionParamsWithInitialSync)
+        val aliceSession2 = mTestHelper.logIntoAccount(aliceUserId, KeysBackupTestConstants.defaultSessionParamsWithInitialSync)
 
         // - Post a message to have a new megolm session
         aliceSession2.cryptoService().setWarnOnUnknownDevices(false)
@@ -1093,7 +1083,7 @@ class KeysBackupTest : InstrumentedTest {
 
         assertFalse(keysBackup.isEnabled)
 
-        val keyBackupCreationInfo = prepareAndCreateKeysBackupData(keysBackup)
+        val keyBackupCreationInfo = mKeysBackupTestHelper.prepareAndCreateKeysBackupData(keysBackup)
 
         assertTrue(keysBackup.isEnabled)
 
@@ -1106,169 +1096,4 @@ class KeysBackupTest : InstrumentedTest {
         stateObserver.stopAndCheckStates(null)
         cryptoTestData.cleanUp(mTestHelper)
     }
-
-    /* ==========================================================================================
-     * Private
-     * ========================================================================================== */
-
-    /**
-     * As KeysBackup is doing asynchronous call to update its internal state, this method help to wait for the
-     * KeysBackup object to be in the specified state
-     */
-    private fun waitForKeysBackupToBeInState(session: Session, state: KeysBackupState) {
-        // If already in the wanted state, return
-        if (session.cryptoService().keysBackupService().state == state) {
-            return
-        }
-
-        // Else observe state changes
-        val latch = CountDownLatch(1)
-
-        session.cryptoService().keysBackupService().addListener(object : KeysBackupStateListener {
-            override fun onStateChange(newState: KeysBackupState) {
-                if (newState == state) {
-                    session.cryptoService().keysBackupService().removeListener(this)
-                    latch.countDown()
-                }
-            }
-        })
-
-        mTestHelper.await(latch)
-    }
-
-    private data class PrepareKeysBackupDataResult(val megolmBackupCreationInfo: MegolmBackupCreationInfo,
-                                                   val version: String)
-
-    private fun prepareAndCreateKeysBackupData(keysBackup: KeysBackupService,
-                                               password: String? = null): PrepareKeysBackupDataResult {
-        val stateObserver = StateObserver(keysBackup)
-
-        val megolmBackupCreationInfo = mTestHelper.doSync<MegolmBackupCreationInfo> {
-            keysBackup.prepareKeysBackupVersion(password, null, it)
-        }
-
-        assertNotNull(megolmBackupCreationInfo)
-
-        assertFalse(keysBackup.isEnabled)
-
-        // Create the version
-        val keysVersion = mTestHelper.doSync<KeysVersion> {
-            keysBackup.createKeysBackupVersion(megolmBackupCreationInfo, it)
-        }
-
-        assertNotNull(keysVersion.version)
-
-        // Backup must be enable now
-        assertTrue(keysBackup.isEnabled)
-
-        stateObserver.stopAndCheckStates(null)
-        return PrepareKeysBackupDataResult(megolmBackupCreationInfo, keysVersion.version!!)
-    }
-
-    private fun assertKeysEquals(keys1: MegolmSessionData?, keys2: MegolmSessionData?) {
-        assertNotNull(keys1)
-        assertNotNull(keys2)
-
-        assertEquals(keys1?.algorithm, keys2?.algorithm)
-        assertEquals(keys1?.roomId, keys2?.roomId)
-        // No need to compare the shortcut
-        // assertEquals(keys1?.sender_claimed_ed25519_key, keys2?.sender_claimed_ed25519_key)
-        assertEquals(keys1?.senderKey, keys2?.senderKey)
-        assertEquals(keys1?.sessionId, keys2?.sessionId)
-        assertEquals(keys1?.sessionKey, keys2?.sessionKey)
-
-        assertListEquals(keys1?.forwardingCurve25519KeyChain, keys2?.forwardingCurve25519KeyChain)
-        assertDictEquals(keys1?.senderClaimedKeys, keys2?.senderClaimedKeys)
-    }
-
-    /**
-     * Data class to store result of [createKeysBackupScenarioWithPassword]
-     */
-    private data class KeysBackupScenarioData(val cryptoTestData: CryptoTestData,
-                                              val aliceKeys: List<OlmInboundGroupSessionWrapper>,
-                                              val prepareKeysBackupDataResult: PrepareKeysBackupDataResult,
-                                              val aliceSession2: Session) {
-        fun cleanUp(testHelper: CommonTestHelper) {
-            cryptoTestData.cleanUp(testHelper)
-            testHelper.signOutAndClose(aliceSession2)
-        }
-    }
-
-    /**
-     * Common initial condition
-     * - Do an e2e backup to the homeserver
-     * - Log Alice on a new device, and wait for its keysBackup object to be ready (in state NotTrusted)
-     *
-     * @param password optional password
-     */
-    private fun createKeysBackupScenarioWithPassword(password: String?): KeysBackupScenarioData {
-        val cryptoTestData = mCryptoTestHelper.doE2ETestWithAliceAndBobInARoomWithEncryptedMessages()
-
-        val cryptoStore = (cryptoTestData.firstSession.cryptoService().keysBackupService() as DefaultKeysBackupService).store
-        val keysBackup = cryptoTestData.firstSession.cryptoService().keysBackupService()
-
-        val stateObserver = StateObserver(keysBackup)
-
-        val aliceKeys = cryptoStore.inboundGroupSessionsToBackup(100)
-
-        // - Do an e2e backup to the homeserver
-        val prepareKeysBackupDataResult = prepareAndCreateKeysBackupData(keysBackup, password)
-
-        var lastProgress = 0
-        var lastTotal = 0
-        mTestHelper.doSync<Unit> {
-            keysBackup.backupAllGroupSessions(object : ProgressListener {
-                override fun onProgress(progress: Int, total: Int) {
-                    lastProgress = progress
-                    lastTotal = total
-                }
-            }, it)
-        }
-
-        assertEquals(2, lastProgress)
-        assertEquals(2, lastTotal)
-
-        val aliceUserId = cryptoTestData.firstSession.myUserId
-
-        // - Log Alice on a new device
-        val aliceSession2 = mTestHelper.logIntoAccount(aliceUserId, defaultSessionParamsWithInitialSync)
-
-        // Test check: aliceSession2 has no keys at login
-        assertEquals(0, aliceSession2.cryptoService().inboundGroupSessionsCount(false))
-
-        // Wait for backup state to be NotTrusted
-        waitForKeysBackupToBeInState(aliceSession2, KeysBackupState.NotTrusted)
-
-        stateObserver.stopAndCheckStates(null)
-
-        return KeysBackupScenarioData(cryptoTestData,
-                aliceKeys,
-                prepareKeysBackupDataResult,
-                aliceSession2)
-    }
-
-    /**
-     * Common restore success check after [createKeysBackupScenarioWithPassword]:
-     * - Imported keys number must be correct
-     * - The new device must have the same count of megolm keys
-     * - Alice must have the same keys on both devices
-     */
-    private fun checkRestoreSuccess(testData: KeysBackupScenarioData,
-                                    total: Int,
-                                    imported: Int) {
-        // - Imported keys number must be correct
-        assertEquals(testData.aliceKeys.size, total)
-        assertEquals(total, imported)
-
-        // - The new device must have the same count of megolm keys
-        assertEquals(testData.aliceKeys.size, testData.aliceSession2.cryptoService().inboundGroupSessionsCount(false))
-
-        // - Alice must have the same keys on both devices
-        for (aliceKey1 in testData.aliceKeys) {
-            val aliceKey2 = (testData.aliceSession2.cryptoService().keysBackupService() as DefaultKeysBackupService).store
-                    .getInboundGroupSession(aliceKey1.olmInboundGroupSession!!.sessionIdentifier(), aliceKey1.senderKey!!)
-            assertNotNull(aliceKey2)
-            assertKeysEquals(aliceKey1.exportKeys(), aliceKey2!!.exportKeys())
-        }
-    }
 }

matrix-sdk-android/src/androidTest/java/im/vector/matrix/android/internal/crypto/keysbackup/KeysBackupTestConstants.kt

@@ -0,0 +1,24 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package im.vector.matrix.android.internal.crypto.keysbackup
+
+import im.vector.matrix.android.common.SessionTestParams
+
+object KeysBackupTestConstants {
+    val defaultSessionParams = SessionTestParams(withInitialSync = false)
+    val defaultSessionParamsWithInitialSync = SessionTestParams(withInitialSync = true)
+}

matrix-sdk-android/src/androidTest/java/im/vector/matrix/android/internal/crypto/keysbackup/KeysBackupTestHelper.kt

@@ -0,0 +1,182 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package im.vector.matrix.android.internal.crypto.keysbackup
+
+import im.vector.matrix.android.api.listeners.ProgressListener
+import im.vector.matrix.android.api.session.Session
+import im.vector.matrix.android.api.session.crypto.keysbackup.KeysBackupService
+import im.vector.matrix.android.api.session.crypto.keysbackup.KeysBackupState
+import im.vector.matrix.android.api.session.crypto.keysbackup.KeysBackupStateListener
+import im.vector.matrix.android.common.CommonTestHelper
+import im.vector.matrix.android.common.CryptoTestHelper
+import im.vector.matrix.android.common.assertDictEquals
+import im.vector.matrix.android.common.assertListEquals
+import im.vector.matrix.android.internal.crypto.MegolmSessionData
+import im.vector.matrix.android.internal.crypto.keysbackup.model.MegolmBackupCreationInfo
+import im.vector.matrix.android.internal.crypto.keysbackup.model.rest.KeysVersion
+import org.junit.Assert
+import java.util.concurrent.CountDownLatch
+
+class KeysBackupTestHelper(
+        private val mTestHelper: CommonTestHelper,
+        private val mCryptoTestHelper: CryptoTestHelper) {
+
+    /**
+     * Common initial condition
+     * - Do an e2e backup to the homeserver
+     * - Log Alice on a new device, and wait for its keysBackup object to be ready (in state NotTrusted)
+     *
+     * @param password optional password
+     */
+    fun createKeysBackupScenarioWithPassword(password: String?): KeysBackupScenarioData {
+        val cryptoTestData = mCryptoTestHelper.doE2ETestWithAliceAndBobInARoomWithEncryptedMessages()
+
+        val cryptoStore = (cryptoTestData.firstSession.cryptoService().keysBackupService() as DefaultKeysBackupService).store
+        val keysBackup = cryptoTestData.firstSession.cryptoService().keysBackupService()
+
+        val stateObserver = StateObserver(keysBackup)
+
+        val aliceKeys = cryptoStore.inboundGroupSessionsToBackup(100)
+
+        // - Do an e2e backup to the homeserver
+        val prepareKeysBackupDataResult = prepareAndCreateKeysBackupData(keysBackup, password)
+
+        var lastProgress = 0
+        var lastTotal = 0
+        mTestHelper.doSync<Unit> {
+            keysBackup.backupAllGroupSessions(object : ProgressListener {
+                override fun onProgress(progress: Int, total: Int) {
+                    lastProgress = progress
+                    lastTotal = total
+                }
+            }, it)
+        }
+
+        Assert.assertEquals(2, lastProgress)
+        Assert.assertEquals(2, lastTotal)
+
+        val aliceUserId = cryptoTestData.firstSession.myUserId
+
+        // - Log Alice on a new device
+        val aliceSession2 = mTestHelper.logIntoAccount(aliceUserId, KeysBackupTestConstants.defaultSessionParamsWithInitialSync)
+
+        // Test check: aliceSession2 has no keys at login
+        Assert.assertEquals(0, aliceSession2.cryptoService().inboundGroupSessionsCount(false))
+
+        // Wait for backup state to be NotTrusted
+        waitForKeysBackupToBeInState(aliceSession2, KeysBackupState.NotTrusted)
+
+        stateObserver.stopAndCheckStates(null)
+
+        return KeysBackupScenarioData(cryptoTestData,
+                aliceKeys,
+                prepareKeysBackupDataResult,
+                aliceSession2)
+    }
+
+    fun prepareAndCreateKeysBackupData(keysBackup: KeysBackupService,
+                                       password: String? = null): PrepareKeysBackupDataResult {
+        val stateObserver = StateObserver(keysBackup)
+
+        val megolmBackupCreationInfo = mTestHelper.doSync<MegolmBackupCreationInfo> {
+            keysBackup.prepareKeysBackupVersion(password, null, it)
+        }
+
+        Assert.assertNotNull(megolmBackupCreationInfo)
+
+        Assert.assertFalse(keysBackup.isEnabled)
+
+        // Create the version
+        val keysVersion = mTestHelper.doSync<KeysVersion> {
+            keysBackup.createKeysBackupVersion(megolmBackupCreationInfo, it)
+        }
+
+        Assert.assertNotNull(keysVersion.version)
+
+        // Backup must be enable now
+        Assert.assertTrue(keysBackup.isEnabled)
+
+        stateObserver.stopAndCheckStates(null)
+        return PrepareKeysBackupDataResult(megolmBackupCreationInfo, keysVersion.version!!)
+    }
+
+    /**
+     * As KeysBackup is doing asynchronous call to update its internal state, this method help to wait for the
+     * KeysBackup object to be in the specified state
+     */
+    fun waitForKeysBackupToBeInState(session: Session, state: KeysBackupState) {
+        // If already in the wanted state, return
+        if (session.cryptoService().keysBackupService().state == state) {
+            return
+        }
+
+        // Else observe state changes
+        val latch = CountDownLatch(1)
+
+        session.cryptoService().keysBackupService().addListener(object : KeysBackupStateListener {
+            override fun onStateChange(newState: KeysBackupState) {
+                if (newState == state) {
+                    session.cryptoService().keysBackupService().removeListener(this)
+                    latch.countDown()
+                }
+            }
+        })
+
+        mTestHelper.await(latch)
+    }
+
+    fun assertKeysEquals(keys1: MegolmSessionData?, keys2: MegolmSessionData?) {
+        Assert.assertNotNull(keys1)
+        Assert.assertNotNull(keys2)
+
+        Assert.assertEquals(keys1?.algorithm, keys2?.algorithm)
+        Assert.assertEquals(keys1?.roomId, keys2?.roomId)
+        // No need to compare the shortcut
+        // assertEquals(keys1?.sender_claimed_ed25519_key, keys2?.sender_claimed_ed25519_key)
+        Assert.assertEquals(keys1?.senderKey, keys2?.senderKey)
+        Assert.assertEquals(keys1?.sessionId, keys2?.sessionId)
+        Assert.assertEquals(keys1?.sessionKey, keys2?.sessionKey)
+
+        assertListEquals(keys1?.forwardingCurve25519KeyChain, keys2?.forwardingCurve25519KeyChain)
+        assertDictEquals(keys1?.senderClaimedKeys, keys2?.senderClaimedKeys)
+    }
+
+    /**
+     * Common restore success check after [KeysBackupTestHelper.createKeysBackupScenarioWithPassword]:
+     * - Imported keys number must be correct
+     * - The new device must have the same count of megolm keys
+     * - Alice must have the same keys on both devices
+     */
+    fun checkRestoreSuccess(testData: KeysBackupScenarioData,
+                            total: Int,
+                            imported: Int) {
+        // - Imported keys number must be correct
+        Assert.assertEquals(testData.aliceKeys.size, total)
+        Assert.assertEquals(total, imported)
+
+        // - The new device must have the same count of megolm keys
+        Assert.assertEquals(testData.aliceKeys.size, testData.aliceSession2.cryptoService().inboundGroupSessionsCount(false))
+
+        // - Alice must have the same keys on both devices
+        for (aliceKey1 in testData.aliceKeys) {
+            val aliceKey2 = (testData.aliceSession2.cryptoService().keysBackupService() as DefaultKeysBackupService).store
+                    .getInboundGroupSession(aliceKey1.olmInboundGroupSession!!.sessionIdentifier(), aliceKey1.senderKey!!)
+            Assert.assertNotNull(aliceKey2)
+            assertKeysEquals(aliceKey1.exportKeys(), aliceKey2!!.exportKeys())
+        }
+    }
+}

matrix-sdk-android/src/androidTest/java/im/vector/matrix/android/internal/crypto/keysbackup/PrepareKeysBackupDataResult.kt

@@ -0,0 +1,22 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package im.vector.matrix.android.internal.crypto.keysbackup
+
+import im.vector.matrix.android.internal.crypto.keysbackup.model.MegolmBackupCreationInfo
+
+data class PrepareKeysBackupDataResult(val megolmBackupCreationInfo: MegolmBackupCreationInfo,
+                                       val version: String)

matrix-sdk-android/src/debug/java/im/vector/matrix/android/internal/network/interceptors/CurlLoggingInterceptor.kt

@@ -20,7 +20,6 @@ package im.vector.matrix.android.internal.network.interceptors
 import im.vector.matrix.android.internal.di.MatrixScope
 import okhttp3.Interceptor
 import okhttp3.Response
-import okhttp3.logging.HttpLoggingInterceptor
 import okio.Buffer
 import timber.log.Timber
 import java.io.IOException
@@ -37,7 +36,7 @@ import javax.inject.Inject
  * non-production environment.
  */
 @MatrixScope
-internal class CurlLoggingInterceptor @Inject constructor(private val logger: HttpLoggingInterceptor.Logger)
+internal class CurlLoggingInterceptor @Inject constructor()
     : Interceptor {
 
     /**
@@ -97,8 +96,8 @@ internal class CurlLoggingInterceptor @Inject constructor(private val logger: Ht
         // Add Json formatting
         curlCmd += " | python -m json.tool"
 
-        logger.log("--- cURL (" + request.url + ")")
-        logger.log(curlCmd)
+        Timber.d("--- cURL (${request.url})")
+        Timber.d(curlCmd)
 
         return chain.proceed(request)
     }

matrix-sdk-android/src/main/java/im/vector/matrix/android/api/auth/AuthenticationService.kt

@@ -23,6 +23,7 @@ import im.vector.matrix.android.api.auth.data.LoginFlowResult
 import im.vector.matrix.android.api.auth.data.SessionParams
 import im.vector.matrix.android.api.auth.login.LoginWizard
 import im.vector.matrix.android.api.auth.registration.RegistrationWizard
+import im.vector.matrix.android.api.auth.wellknown.WellknownResult
 import im.vector.matrix.android.api.session.Session
 import im.vector.matrix.android.api.util.Cancelable
 
@@ -30,7 +31,6 @@ import im.vector.matrix.android.api.util.Cancelable
  * This interface defines methods to authenticate or to create an account to a matrix server.
  */
 interface AuthenticationService {
-
     /**
      * Request the supported login flows for this homeserver.
      * This is the first method to call to be able to get a wizard to login or the create an account
@@ -89,4 +89,20 @@ interface AuthenticationService {
     fun createSessionFromSso(homeServerConnectionConfig: HomeServerConnectionConfig,
                              credentials: Credentials,
                              callback: MatrixCallback<Session>): Cancelable
+
+    /**
+     * Perform a wellknown request, using the domain from the matrixId
+     */
+    fun getWellKnownData(matrixId: String,
+                         callback: MatrixCallback<WellknownResult>): Cancelable
+
+    /**
+     * Authenticate with a matrixId and a password
+     * Usually call this after a successful call to getWellKnownData()
+     */
+    fun directAuthentication(homeServerConnectionConfig: HomeServerConnectionConfig,
+                             matrixId: String,
+                             password: String,
+                             initialDeviceName: String,
+                             callback: MatrixCallback<Session>): Cancelable
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/api/auth/Constants.kt

@@ -0,0 +1,37 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package im.vector.matrix.android.api.auth
+
+/**
+ * Path to use when the client does not supported any or all login flows
+ * Ref: https://matrix.org/docs/spec/client_server/latest#login-fallback
+ * */
+const val LOGIN_FALLBACK_PATH = "/_matrix/static/client/login/"
+
+/**
+ * Path to use when the client does not supported any or all registration flows
+ * Not documented
+ */
+const val REGISTER_FALLBACK_PATH = "/_matrix/static/client/register/"
+
+/**
+ * Path to use when the client want to connect using SSO
+ * Ref: https://matrix.org/docs/spec/client_server/latest#sso-client-login
+ */
+const val SSO_FALLBACK_PATH = "/_matrix/client/r0/login/sso/redirect"
+
+const val SSO_REDIRECT_URL_PARAM = "redirectUrl"

matrix-sdk-android/src/main/java/im/vector/matrix/android/api/auth/data/Credentials.kt

@@ -24,16 +24,38 @@ import im.vector.matrix.android.internal.util.md5
  * This data class hold credentials user data.
  * You shouldn't have to instantiate it.
  * The access token should be use to authenticate user in all server requests.
+ * Ref: https://matrix.org/docs/spec/client_server/latest#post-matrix-client-r0-login
  */
 @JsonClass(generateAdapter = true)
 data class Credentials(
+        /**
+         * The fully-qualified Matrix ID that has been registered.
+         */
         @Json(name = "user_id") val userId: String,
-        @Json(name = "home_server") val homeServer: String,
+        /**
+         * An access token for the account. This access token can then be used to authorize other requests.
+         */
         @Json(name = "access_token") val accessToken: String,
+        /**
+         * Not documented
+         */
         @Json(name = "refresh_token") val refreshToken: String?,
+        /**
+         * The server_name of the homeserver on which the account has been registered.
+         * @Deprecated. Clients should extract the server_name from user_id (by splitting at the first colon)
+         * if they require it. Note also that homeserver is not spelt this way.
+         */
+        @Json(name = "home_server") val homeServer: String,
+        /**
+         * ID of the logged-in device. Will be the same as the corresponding parameter in the request, if one was specified.
+         */
         @Json(name = "device_id") val deviceId: String?,
-        // Optional data that may contain info to override home server and/or identity server
-        @Json(name = "well_known") val wellKnown: WellKnown? = null
+        /**
+         * Optional client configuration provided by the server. If present, clients SHOULD use the provided object to
+         * reconfigure themselves, optionally validating the URLs within.
+         * This object takes the same form as the one returned from .well-known autodiscovery.
+         */
+        @Json(name = "well_known") val discoveryInformation: DiscoveryInformation? = null
 )
 
 internal fun Credentials.sessionId(): String {

matrix-sdk-android/src/main/java/im/vector/matrix/android/api/auth/data/DiscoveryInformation.kt

@@ -0,0 +1,40 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package im.vector.matrix.android.api.auth.data
+
+import com.squareup.moshi.Json
+import com.squareup.moshi.JsonClass
+
+/**
+ * This is a light version of Wellknown model, used for login response
+ * Ref: https://matrix.org/docs/spec/client_server/latest#post-matrix-client-r0-login
+ */
+@JsonClass(generateAdapter = true)
+data class DiscoveryInformation(
+        /**
+         * Required. Used by clients to discover homeserver information.
+         */
+        @Json(name = "m.homeserver")
+        val homeServer: WellKnownBaseConfig? = null,
+
+        /**
+         * Used by clients to discover identity server information.
+         * Note: matrix.org does not send this field
+         */
+        @Json(name = "m.identity_server")
+        val identityServer: WellKnownBaseConfig? = null
+)

matrix-sdk-android/src/main/java/im/vector/matrix/android/api/auth/data/WellKnown.kt

@@ -18,6 +18,7 @@ package im.vector.matrix.android.api.auth.data
 
 import com.squareup.moshi.Json
 import com.squareup.moshi.JsonClass
+import im.vector.matrix.android.api.util.JsonDict
 
 /**
  * https://matrix.org/docs/spec/client_server/r0.4.0.html#server-discovery
@@ -52,7 +53,7 @@ data class WellKnown(
         val identityServer: WellKnownBaseConfig? = null,
 
         @Json(name = "m.integrations")
-        val integrations: Map<String, @JvmSuppressWildcards Any>? = null
+        val integrations: JsonDict? = null
 ) {
     /**
      * Returns the list of integration managers proposed

matrix-sdk-android/src/main/java/im/vector/matrix/android/api/auth/data/WellKnownManagerConfig.kt

@@ -16,6 +16,6 @@
 package im.vector.matrix.android.api.auth.data
 
 data class WellKnownManagerConfig(
-        val apiUrl : String,
+        val apiUrl: String,
         val uiUrl: String
 )

matrix-sdk-android/src/main/java/im/vector/matrix/android/api/auth/wellknown/WellknownResult.kt

@@ -0,0 +1,55 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package im.vector.matrix.android.api.auth.wellknown
+
+import im.vector.matrix.android.api.auth.data.WellKnown
+
+/**
+ * Ref: https://matrix.org/docs/spec/client_server/latest#well-known-uri
+ */
+sealed class WellknownResult {
+    /**
+     * The provided matrixId is no valid. Unable to extract a domain name.
+     */
+    object InvalidMatrixId : WellknownResult()
+
+    /**
+     * Retrieve the specific piece of information from the user in a way which fits within the existing client user experience,
+     * if the client is inclined to do so. Failure can take place instead if no good user experience for this is possible at this point.
+     */
+    data class Prompt(val homeServerUrl: String,
+                      val identityServerUrl: String?,
+                      val wellKnown: WellKnown) : WellknownResult()
+
+    /**
+     * Stop the current auto-discovery mechanism. If no more auto-discovery mechanisms are available,
+     * then the client may use other methods of determining the required parameters, such as prompting the user, or using default values.
+     */
+    object Ignore : WellknownResult()
+
+    /**
+     * Inform the user that auto-discovery failed due to invalid/empty data and PROMPT for the parameter.
+     */
+    object FailPrompt : WellknownResult()
+
+    /**
+     * Inform the user that auto-discovery did not return any usable URLs. Do not continue further with the current login process.
+     * At this point, valid data was obtained, but no homeserver is available to serve the client.
+     * No further guess should be attempted and the user should make a conscientious decision what to do next.
+     */
+    object FailError : WellknownResult()
+}

matrix-sdk-android/src/main/java/im/vector/matrix/android/api/failure/Extensions.kt

@@ -16,6 +16,10 @@
 
 package im.vector.matrix.android.api.failure
 
+import im.vector.matrix.android.api.extensions.tryThis
+import im.vector.matrix.android.internal.auth.registration.RegistrationFlowResponse
+import im.vector.matrix.android.internal.di.MoshiProvider
+import java.io.IOException
 import javax.net.ssl.HttpsURLConnection
 
 fun Throwable.is401() =
@@ -29,6 +33,7 @@ fun Throwable.isTokenError() =
 
 fun Throwable.shouldBeRetried(): Boolean {
     return this is Failure.NetworkConnection
+            || this is IOException
             || (this is Failure.ServerError && error.code == MatrixError.M_LIMIT_EXCEEDED)
 }
 
@@ -37,3 +42,18 @@ fun Throwable.isInvalidPassword(): Boolean {
             && error.code == MatrixError.M_FORBIDDEN
             && error.message == "Invalid password"
 }
+
+/**
+ * Try to convert to a RegistrationFlowResponse. Return null in the cases it's not possible
+ */
+fun Throwable.toRegistrationFlowResponse(): RegistrationFlowResponse? {
+    return if (this is Failure.OtherServerError && this.httpCode == 401) {
+        tryThis {
+            MoshiProvider.providesMoshi()
+                    .adapter(RegistrationFlowResponse::class.java)
+                    .fromJson(this.errorBody)
+        }
+    } else {
+        null
+    }
+}

matrix-sdk-android/src/main/java/im/vector/matrix/android/api/session/account/AccountService.kt

@@ -23,11 +23,28 @@ import im.vector.matrix.android.api.util.Cancelable
  * This interface defines methods to manage the account. It's implemented at the session level.
  */
 interface AccountService {
-
     /**
      * Ask the homeserver to change the password.
      * @param password Current password.
      * @param newPassword New password
      */
     fun changePassword(password: String, newPassword: String, callback: MatrixCallback<Unit>): Cancelable
+
+    /**
+     * Deactivate the account.
+     *
+     * This will make your account permanently unusable. You will not be able to log in, and no one will be able to re-register
+     * the same user ID. This will cause your account to leave all rooms it is participating in, and it will remove your account
+     * details from your identity server. <b>This action is irreversible</b>.\n\nDeactivating your account <b>does not by default
+     * cause us to forget messages you have sent</b>. If you would like us to forget your messages, please tick the box below.
+     *
+     * Message visibility in Matrix is similar to email. Our forgetting your messages means that messages you have sent will not
+     * be shared with any new or unregistered users, but registered users who already have access to these messages will still
+     * have access to their copy.
+     *
+     * @param password the account password
+     * @param eraseAllData set to true to forget all messages that have been sent. Warning: this will cause future users to see
+     * an incomplete view of conversations
+     */
+    fun deactivateAccount(password: String, eraseAllData: Boolean, callback: MatrixCallback<Unit>): Cancelable
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/api/session/crypto/CryptoService.kt

@@ -98,7 +98,9 @@ interface CryptoService {
 
     fun removeRoomKeysRequestListener(listener: GossipingRequestListener)
 
-    fun getDevicesList(callback: MatrixCallback<DevicesListResponse>)
+    fun fetchDevicesList(callback: MatrixCallback<DevicesListResponse>)
+    fun getMyDevicesInfo() : List<DeviceInfo>
+    fun getLiveMyDevicesInfo() : LiveData<List<DeviceInfo>>
 
     fun getDeviceInfo(deviceId: String, callback: MatrixCallback<DeviceInfo>)
 
@@ -111,6 +113,8 @@ interface CryptoService {
                             roomId: String,
                             callback: MatrixCallback<MXEncryptEventContentResult>)
 
+    fun discardOutboundSession(roomId: String)
+
     @Throws(MXCryptoError::class)
     fun decryptEvent(event: Event, timeline: String): MXEventDecryptionResult
 

matrix-sdk-android/src/main/java/im/vector/matrix/android/api/session/crypto/crosssigning/CrossSigningService.kt

@@ -55,6 +55,8 @@ interface CrossSigningService {
 
     fun getCrossSigningPrivateKeys(): PrivateKeysInfo?
 
+    fun getLiveCrossSigningPrivateKeys(): LiveData<Optional<PrivateKeysInfo>>
+
     fun canCrossSign(): Boolean
 
     fun trustUser(otherUserId: String,

matrix-sdk-android/src/main/java/im/vector/matrix/android/api/session/events/model/EventType.kt

@@ -81,6 +81,9 @@ object EventType {
     // Relation Events
     const val REACTION = "m.reaction"
 
+    // Unwedging
+    internal const val DUMMY = "m.dummy"
+
     private val STATE_EVENTS = listOf(
             STATE_ROOM_NAME,
             STATE_ROOM_TOPIC,

matrix-sdk-android/src/main/java/im/vector/matrix/android/api/session/room/model/RoomSummary.kt

@@ -46,10 +46,10 @@ data class RoomSummary constructor(
         val readMarkerId: String? = null,
         val userDrafts: List<UserDraft> = emptyList(),
         val isEncrypted: Boolean,
+        val encryptionEventTs: Long?,
         val inviterId: String? = null,
         val typingRoomMemberIds: List<String> = emptyList(),
         val breadcrumbsIndex: Int = NOT_IN_BREADCRUMBS,
-        // TODO Plug it
         val roomEncryptionTrustLevel: RoomEncryptionTrustLevel? = null
 ) {
 

matrix-sdk-android/src/main/java/im/vector/matrix/android/api/session/room/timeline/Timeline.kt

@@ -104,6 +104,7 @@ interface Timeline {
     interface Listener {
         /**
          * Call when the timeline has been updated through pagination or sync.
+         * The latest event is the first in the list
          * @param snapshot the most up to date snapshot
          */
         fun onTimelineUpdated(snapshot: List<TimelineEvent>)

matrix-sdk-android/src/main/java/im/vector/matrix/android/api/session/user/UserService.kt

@@ -61,9 +61,10 @@ interface UserService {
     /**
      * Observe a live [PagedList] of users sorted alphabetically. You can filter the users.
      * @param filter the filter. It will look into userId and displayName.
+     * @param excludedUserIds userId list which will be excluded from the result list.
      * @return a Livedata of users
      */
-    fun getPagedUsersLive(filter: String? = null): LiveData<PagedList<User>>
+    fun getPagedUsersLive(filter: String? = null, excludedUserIds: Set<String>? = null): LiveData<PagedList<User>>
 
     /**
      * Get list of ignored users

matrix-sdk-android/src/main/java/im/vector/matrix/android/api/session/user/model/User.kt

@@ -22,6 +22,9 @@ package im.vector.matrix.android.api.session.user.model
  */
 data class User(
         val userId: String,
+        /**
+         * For usage in UI, consider using [getBestName]
+         */
         val displayName: String? = null,
         val avatarUrl: String? = null
 ) {

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/auth/AuthModule.kt

@@ -25,6 +25,10 @@ import im.vector.matrix.android.internal.auth.db.AuthRealmMigration
 import im.vector.matrix.android.internal.auth.db.AuthRealmModule
 import im.vector.matrix.android.internal.auth.db.RealmPendingSessionStore
 import im.vector.matrix.android.internal.auth.db.RealmSessionParamsStore
+import im.vector.matrix.android.internal.auth.wellknown.DefaultDirectLoginTask
+import im.vector.matrix.android.internal.auth.wellknown.DefaultGetWellknownTask
+import im.vector.matrix.android.internal.auth.wellknown.DirectLoginTask
+import im.vector.matrix.android.internal.auth.wellknown.GetWellknownTask
 import im.vector.matrix.android.internal.database.RealmKeysUtils
 import im.vector.matrix.android.internal.di.AuthDatabase
 import io.realm.RealmConfiguration
@@ -59,14 +63,20 @@ internal abstract class AuthModule {
     }
 
     @Binds
-    abstract fun bindSessionParamsStore(sessionParamsStore: RealmSessionParamsStore): SessionParamsStore
+    abstract fun bindSessionParamsStore(store: RealmSessionParamsStore): SessionParamsStore
 
     @Binds
-    abstract fun bindPendingSessionStore(pendingSessionStore: RealmPendingSessionStore): PendingSessionStore
+    abstract fun bindPendingSessionStore(store: RealmPendingSessionStore): PendingSessionStore
 
     @Binds
-    abstract fun bindAuthenticationService(authenticationService: DefaultAuthenticationService): AuthenticationService
+    abstract fun bindAuthenticationService(service: DefaultAuthenticationService): AuthenticationService
 
     @Binds
-    abstract fun bindSessionCreator(sessionCreator: DefaultSessionCreator): SessionCreator
+    abstract fun bindSessionCreator(creator: DefaultSessionCreator): SessionCreator
+
+    @Binds
+    abstract fun bindGetWellknownTask(task: DefaultGetWellknownTask): GetWellknownTask
+
+    @Binds
+    abstract fun bindDirectLoginTask(task: DefaultDirectLoginTask): DirectLoginTask
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/auth/DefaultAuthenticationService.kt

@@ -29,6 +29,7 @@ import im.vector.matrix.android.api.auth.data.isLoginAndRegistrationSupportedByS
 import im.vector.matrix.android.api.auth.data.isSupportedBySdk
 import im.vector.matrix.android.api.auth.login.LoginWizard
 import im.vector.matrix.android.api.auth.registration.RegistrationWizard
+import im.vector.matrix.android.api.auth.wellknown.WellknownResult
 import im.vector.matrix.android.api.failure.Failure
 import im.vector.matrix.android.api.session.Session
 import im.vector.matrix.android.api.util.Cancelable
@@ -38,11 +39,16 @@ import im.vector.matrix.android.internal.auth.data.RiotConfig
 import im.vector.matrix.android.internal.auth.db.PendingSessionData
 import im.vector.matrix.android.internal.auth.login.DefaultLoginWizard
 import im.vector.matrix.android.internal.auth.registration.DefaultRegistrationWizard
+import im.vector.matrix.android.internal.auth.wellknown.DirectLoginTask
+import im.vector.matrix.android.internal.auth.wellknown.GetWellknownTask
 import im.vector.matrix.android.internal.di.Unauthenticated
 import im.vector.matrix.android.internal.network.RetrofitFactory
 import im.vector.matrix.android.internal.network.executeRequest
+import im.vector.matrix.android.internal.task.TaskExecutor
+import im.vector.matrix.android.internal.task.configureWith
 import im.vector.matrix.android.internal.task.launchToCallback
 import im.vector.matrix.android.internal.util.MatrixCoroutineDispatchers
+import im.vector.matrix.android.internal.util.exhaustive
 import im.vector.matrix.android.internal.util.toCancelable
 import kotlinx.coroutines.GlobalScope
 import kotlinx.coroutines.launch
@@ -59,7 +65,10 @@ internal class DefaultAuthenticationService @Inject constructor(
         private val sessionParamsStore: SessionParamsStore,
         private val sessionManager: SessionManager,
         private val sessionCreator: SessionCreator,
-        private val pendingSessionStore: PendingSessionStore
+        private val pendingSessionStore: PendingSessionStore,
+        private val getWellknownTask: GetWellknownTask,
+        private val directLoginTask: DirectLoginTask,
+        private val taskExecutor: TaskExecutor
 ) : AuthenticationService {
 
     private var pendingSessionData: PendingSessionData? = pendingSessionStore.getPendingSessionData()
@@ -148,27 +157,71 @@ internal class DefaultAuthenticationService @Inject constructor(
         val authAPI = buildAuthAPI(homeServerConnectionConfig)
 
         // Ok, try to get the config.json file of a RiotWeb client
-        val riotConfig = executeRequest<RiotConfig>(null) {
-            apiCall = authAPI.getRiotConfig()
-        }
-
-        if (riotConfig.defaultHomeServerUrl?.isNotBlank() == true) {
-            // Ok, good sign, we got a default hs url
-            val newHomeServerConnectionConfig = homeServerConnectionConfig.copy(
-                    homeServerUri = Uri.parse(riotConfig.defaultHomeServerUrl)
-            )
-
-            val newAuthAPI = buildAuthAPI(newHomeServerConnectionConfig)
-
-            val versions = executeRequest<Versions>(null) {
-                apiCall = newAuthAPI.versions()
+        return runCatching {
+            executeRequest<RiotConfig>(null) {
+                apiCall = authAPI.getRiotConfig()
             }
-
-            return getLoginFlowResult(newAuthAPI, versions, riotConfig.defaultHomeServerUrl)
-        } else {
-            // Config exists, but there is no default homeserver url (ex: https://riot.im/app)
-            throw Failure.OtherServerError("", HttpsURLConnection.HTTP_NOT_FOUND /* 404 */)
         }
+                .map { riotConfig ->
+                    if (riotConfig.defaultHomeServerUrl?.isNotBlank() == true) {
+                        // Ok, good sign, we got a default hs url
+                        val newHomeServerConnectionConfig = homeServerConnectionConfig.copy(
+                                homeServerUri = Uri.parse(riotConfig.defaultHomeServerUrl)
+                        )
+
+                        val newAuthAPI = buildAuthAPI(newHomeServerConnectionConfig)
+
+                        val versions = executeRequest<Versions>(null) {
+                            apiCall = newAuthAPI.versions()
+                        }
+
+                        getLoginFlowResult(newAuthAPI, versions, riotConfig.defaultHomeServerUrl)
+                    } else {
+                        // Config exists, but there is no default homeserver url (ex: https://riot.im/app)
+                        throw Failure.OtherServerError("", HttpsURLConnection.HTTP_NOT_FOUND /* 404 */)
+                    }
+                }
+                .fold(
+                        {
+                            it
+                        },
+                        {
+                            if (it is Failure.OtherServerError
+                                    && it.httpCode == HttpsURLConnection.HTTP_NOT_FOUND /* 404 */) {
+                                // Try with wellknown
+                                getWellknownLoginFlowInternal(homeServerConnectionConfig)
+                            } else {
+                                throw it
+                            }
+                        }
+                )
+    }
+
+    private suspend fun getWellknownLoginFlowInternal(homeServerConnectionConfig: HomeServerConnectionConfig): LoginFlowResult {
+        val domain = homeServerConnectionConfig.homeServerUri.host
+                ?: throw Failure.OtherServerError("", HttpsURLConnection.HTTP_NOT_FOUND /* 404 */)
+
+        // Create a fake userId, for the getWellknown task
+        val fakeUserId = "@alice:$domain"
+        val wellknownResult = getWellknownTask.execute(GetWellknownTask.Params(fakeUserId))
+
+        return when (wellknownResult) {
+            is WellknownResult.Prompt -> {
+                val newHomeServerConnectionConfig = homeServerConnectionConfig.copy(
+                        homeServerUri = Uri.parse(wellknownResult.homeServerUrl),
+                        identityServerUri = wellknownResult.identityServerUrl?.let { Uri.parse(it) }
+                )
+
+                val newAuthAPI = buildAuthAPI(newHomeServerConnectionConfig)
+
+                val versions = executeRequest<Versions>(null) {
+                    apiCall = newAuthAPI.versions()
+                }
+
+                getLoginFlowResult(newAuthAPI, versions, wellknownResult.homeServerUrl)
+            }
+            else                      -> throw Failure.OtherServerError("", HttpsURLConnection.HTTP_NOT_FOUND /* 404 */)
+        }.exhaustive
     }
 
     private suspend fun getLoginFlowResult(authAPI: AuthAPI, versions: Versions, homeServerUrl: String): LoginFlowResult {
@@ -260,6 +313,26 @@ internal class DefaultAuthenticationService @Inject constructor(
         }
     }
 
+    override fun getWellKnownData(matrixId: String, callback: MatrixCallback<WellknownResult>): Cancelable {
+        return getWellknownTask
+                .configureWith(GetWellknownTask.Params(matrixId)) {
+                    this.callback = callback
+                }
+                .executeBy(taskExecutor)
+    }
+
+    override fun directAuthentication(homeServerConnectionConfig: HomeServerConnectionConfig,
+                                      matrixId: String,
+                                      password: String,
+                                      initialDeviceName: String,
+                                      callback: MatrixCallback<Session>): Cancelable {
+        return directLoginTask
+                .configureWith(DirectLoginTask.Params(homeServerConnectionConfig, matrixId, password, initialDeviceName)) {
+                    this.callback = callback
+                }
+                .executeBy(taskExecutor)
+    }
+
     private suspend fun createSessionFromSso(credentials: Credentials,
                                              homeServerConnectionConfig: HomeServerConnectionConfig): Session = withContext(coroutineDispatchers.computation) {
         sessionCreator.createSession(credentials, homeServerConnectionConfig)

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/auth/SessionCreator.kt

@@ -46,14 +46,14 @@ internal class DefaultSessionCreator @Inject constructor(
         val sessionParams = SessionParams(
                 credentials = credentials,
                 homeServerConnectionConfig = homeServerConnectionConfig.copy(
-                        homeServerUri = credentials.wellKnown?.homeServer?.baseURL
+                        homeServerUri = credentials.discoveryInformation?.homeServer?.baseURL
                                 // remove trailing "/"
                                 ?.trim { it == '/' }
                                 ?.takeIf { it.isNotBlank() }
                                 ?.also { Timber.d("Overriding homeserver url to $it") }
                                 ?.let { Uri.parse(it) }
                                 ?: homeServerConnectionConfig.homeServerUri,
-                        identityServerUri = credentials.wellKnown?.identityServer?.baseURL
+                        identityServerUri = credentials.discoveryInformation?.identityServer?.baseURL
                                 // remove trailing "/"
                                 ?.trim { it == '/' }
                                 ?.takeIf { it.isNotBlank() }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/auth/registration/RegisterTask.kt

@@ -18,8 +18,8 @@ package im.vector.matrix.android.internal.auth.registration
 
 import im.vector.matrix.android.api.auth.data.Credentials
 import im.vector.matrix.android.api.failure.Failure
+import im.vector.matrix.android.api.failure.toRegistrationFlowResponse
 import im.vector.matrix.android.internal.auth.AuthAPI
-import im.vector.matrix.android.internal.di.MoshiProvider
 import im.vector.matrix.android.internal.network.executeRequest
 import im.vector.matrix.android.internal.task.Task
 
@@ -39,25 +39,9 @@ internal class DefaultRegisterTask(
                 apiCall = authAPI.register(params.registrationParams)
             }
         } catch (throwable: Throwable) {
-            if (throwable is Failure.OtherServerError && throwable.httpCode == 401) {
-                // Parse to get a RegistrationFlowResponse
-                val registrationFlowResponse = try {
-                    MoshiProvider.providesMoshi()
-                            .adapter(RegistrationFlowResponse::class.java)
-                            .fromJson(throwable.errorBody)
-                } catch (e: Exception) {
-                    null
-                }
-                // check if the server response can be cast
-                if (registrationFlowResponse != null) {
-                    throw Failure.RegistrationFlowError(registrationFlowResponse)
-                } else {
-                    throw throwable
-                }
-            } else {
-                // Other error
-                throw throwable
-            }
+            throw throwable.toRegistrationFlowResponse()
+                    ?.let { Failure.RegistrationFlowError(it) }
+                    ?: throwable
         }
     }
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/auth/wellknown/DirectLoginTask.kt

@@ -0,0 +1,61 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package im.vector.matrix.android.internal.auth.wellknown
+
+import dagger.Lazy
+import im.vector.matrix.android.api.auth.data.Credentials
+import im.vector.matrix.android.api.auth.data.HomeServerConnectionConfig
+import im.vector.matrix.android.api.session.Session
+import im.vector.matrix.android.internal.auth.AuthAPI
+import im.vector.matrix.android.internal.auth.SessionCreator
+import im.vector.matrix.android.internal.auth.data.PasswordLoginParams
+import im.vector.matrix.android.internal.di.Unauthenticated
+import im.vector.matrix.android.internal.network.RetrofitFactory
+import im.vector.matrix.android.internal.network.executeRequest
+import im.vector.matrix.android.internal.task.Task
+import okhttp3.OkHttpClient
+import javax.inject.Inject
+
+internal interface DirectLoginTask : Task<DirectLoginTask.Params, Session> {
+    data class Params(
+            val homeServerConnectionConfig: HomeServerConnectionConfig,
+            val userId: String,
+            val password: String,
+            val deviceName: String
+    )
+}
+
+internal class DefaultDirectLoginTask @Inject constructor(
+        @Unauthenticated
+        private val okHttpClient: Lazy<OkHttpClient>,
+        private val retrofitFactory: RetrofitFactory,
+        private val sessionCreator: SessionCreator
+) : DirectLoginTask {
+
+    override suspend fun execute(params: DirectLoginTask.Params): Session {
+        val authAPI = retrofitFactory.create(okHttpClient, params.homeServerConnectionConfig.homeServerUri.toString())
+                .create(AuthAPI::class.java)
+
+        val loginParams = PasswordLoginParams.userIdentifier(params.userId, params.password, params.deviceName)
+
+        val credentials = executeRequest<Credentials>(null) {
+            apiCall = authAPI.login(loginParams)
+        }
+
+        return sessionCreator.createSession(credentials, params.homeServerConnectionConfig)
+    }
+}

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/auth/wellknown/GetWellknownTask.kt

@@ -0,0 +1,199 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package im.vector.matrix.android.internal.auth.wellknown
+
+import android.util.MalformedJsonException
+import dagger.Lazy
+import im.vector.matrix.android.api.MatrixPatterns
+import im.vector.matrix.android.api.auth.data.WellKnown
+import im.vector.matrix.android.api.auth.wellknown.WellknownResult
+import im.vector.matrix.android.api.failure.Failure
+import im.vector.matrix.android.internal.di.Unauthenticated
+import im.vector.matrix.android.internal.identity.IdentityPingApi
+import im.vector.matrix.android.internal.network.RetrofitFactory
+import im.vector.matrix.android.internal.network.executeRequest
+import im.vector.matrix.android.internal.session.homeserver.CapabilitiesAPI
+import im.vector.matrix.android.internal.task.Task
+import im.vector.matrix.android.internal.util.isValidUrl
+import okhttp3.OkHttpClient
+import java.io.EOFException
+import javax.inject.Inject
+import javax.net.ssl.HttpsURLConnection
+
+internal interface GetWellknownTask : Task<GetWellknownTask.Params, WellknownResult> {
+    data class Params(
+            val matrixId: String
+    )
+}
+
+/**
+ * Inspired from AutoDiscovery class from legacy Matrix Android SDK
+ */
+internal class DefaultGetWellknownTask @Inject constructor(
+        @Unauthenticated
+        private val okHttpClient: Lazy<OkHttpClient>,
+        private val retrofitFactory: RetrofitFactory
+) : GetWellknownTask {
+
+    override suspend fun execute(params: GetWellknownTask.Params): WellknownResult {
+        if (!MatrixPatterns.isUserId(params.matrixId)) {
+            return WellknownResult.InvalidMatrixId
+        }
+
+        val homeServerDomain = params.matrixId.substringAfter(":")
+
+        return findClientConfig(homeServerDomain)
+    }
+
+    /**
+     * Find client config
+     *
+     * - Do the .well-known request
+     * - validate homeserver url and identity server url if provide in .well-known result
+     * - return action and .well-known data
+     *
+     * @param domain: homeserver domain, deduced from mx userId (ex: "matrix.org" from userId "@user:matrix.org")
+     */
+    private suspend fun findClientConfig(domain: String): WellknownResult {
+        val wellKnownAPI = retrofitFactory.create(okHttpClient, "https://dummy.org")
+                .create(WellKnownAPI::class.java)
+
+        return try {
+            val wellKnown = executeRequest<WellKnown>(null) {
+                apiCall = wellKnownAPI.getWellKnown(domain)
+            }
+
+            // Success
+            val homeServerBaseUrl = wellKnown.homeServer?.baseURL
+            if (homeServerBaseUrl.isNullOrBlank()) {
+                WellknownResult.FailPrompt
+            } else {
+                if (homeServerBaseUrl.isValidUrl()) {
+                    // Check that HS is a real one
+                    validateHomeServer(homeServerBaseUrl, wellKnown)
+                } else {
+                    WellknownResult.FailError
+                }
+            }
+        } catch (throwable: Throwable) {
+            when (throwable) {
+                is Failure.NetworkConnection               -> {
+                    WellknownResult.Ignore
+                }
+                is Failure.OtherServerError                -> {
+                    when (throwable.httpCode) {
+                        HttpsURLConnection.HTTP_NOT_FOUND -> WellknownResult.Ignore
+                        else                              -> WellknownResult.FailPrompt
+                    }
+                }
+                is MalformedJsonException, is EOFException -> {
+                    WellknownResult.FailPrompt
+                }
+                else                                       -> {
+                    throw throwable
+                }
+            }
+        }
+    }
+
+    /**
+     * Return true if home server is valid, and (if applicable) if identity server is pingable
+     */
+    private suspend fun validateHomeServer(homeServerBaseUrl: String, wellKnown: WellKnown): WellknownResult {
+        val capabilitiesAPI = retrofitFactory.create(okHttpClient, homeServerBaseUrl)
+                .create(CapabilitiesAPI::class.java)
+
+        try {
+            executeRequest<Unit>(null) {
+                apiCall = capabilitiesAPI.getVersions()
+            }
+        } catch (throwable: Throwable) {
+            return WellknownResult.FailError
+        }
+
+        return if (wellKnown.identityServer == null) {
+            // No identity server
+            WellknownResult.Prompt(homeServerBaseUrl, null, wellKnown)
+        } else {
+            // if m.identity_server is present it must be valid
+            val identityServerBaseUrl = wellKnown.identityServer.baseURL
+            if (identityServerBaseUrl.isNullOrBlank()) {
+                WellknownResult.FailError
+            } else {
+                if (identityServerBaseUrl.isValidUrl()) {
+                    if (validateIdentityServer(identityServerBaseUrl)) {
+                        // All is ok
+                        WellknownResult.Prompt(homeServerBaseUrl, identityServerBaseUrl, wellKnown)
+                    } else {
+                        WellknownResult.FailError
+                    }
+                } else {
+                    WellknownResult.FailError
+                }
+            }
+        }
+    }
+
+    /**
+     * Return true if identity server is pingable
+     */
+    private suspend fun validateIdentityServer(identityServerBaseUrl: String): Boolean {
+        val identityPingApi = retrofitFactory.create(okHttpClient, identityServerBaseUrl)
+                .create(IdentityPingApi::class.java)
+
+        return try {
+            executeRequest<Unit>(null) {
+                apiCall = identityPingApi.ping()
+            }
+
+            true
+        } catch (throwable: Throwable) {
+            false
+        }
+    }
+
+    /**
+     * Try to get an identity server URL from a home server URL, using a .wellknown request
+     */
+    /*
+    fun getIdentityServer(homeServerUrl: String, callback: ApiCallback<String?>) {
+        if (homeServerUrl.startsWith("https://")) {
+            wellKnownRestClient.getWellKnown(homeServerUrl.substring("https://".length),
+                    object : SimpleApiCallback<WellKnown>(callback) {
+                        override fun onSuccess(info: WellKnown) {
+                            callback.onSuccess(info.identityServer?.baseURL)
+                        }
+                    })
+        } else {
+            callback.onUnexpectedError(InvalidParameterException("malformed url"))
+        }
+    }
+
+    fun getServerPreferredIntegrationManagers(homeServerUrl: String, callback: ApiCallback<List<WellKnownManagerConfig>>) {
+        if (homeServerUrl.startsWith("https://")) {
+            wellKnownRestClient.getWellKnown(homeServerUrl.substring("https://".length),
+                    object : SimpleApiCallback<WellKnown>(callback) {
+                        override fun onSuccess(info: WellKnown) {
+                            callback.onSuccess(info.getIntegrationManagers())
+                        }
+                    })
+        } else {
+            callback.onUnexpectedError(InvalidParameterException("malformed url"))
+        }
+    }
+     */
+}

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/auth/wellknown/WellKnownAPI.kt

@@ -0,0 +1,26 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package im.vector.matrix.android.internal.auth.wellknown
+
+import im.vector.matrix.android.api.auth.data.WellKnown
+import retrofit2.Call
+import retrofit2.http.GET
+import retrofit2.http.Path
+
+internal interface WellKnownAPI {
+    @GET("https://{domain}/.well-known/matrix/client")
+    fun getWellKnown(@Path("domain") domain: String): Call<WellKnown>
+}

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/CryptoModule.kt

@@ -112,6 +112,7 @@ internal abstract class CryptoModule {
         @SessionScope
         fun providesRealmConfiguration(@SessionFilesDirectory directory: File,
                                        @UserMd5 userMd5: String,
+                                       realmCryptoStoreMigration: RealmCryptoStoreMigration,
                                        realmKeysUtils: RealmKeysUtils): RealmConfiguration {
             return RealmConfiguration.Builder()
                     .directory(directory)
@@ -121,7 +122,7 @@ internal abstract class CryptoModule {
                     .name("crypto_store.realm")
                     .modules(RealmCryptoStoreModule())
                     .schemaVersion(RealmCryptoStoreMigration.CRYPTO_STORE_SCHEMA_VERSION)
-                    .migration(RealmCryptoStoreMigration)
+                    .migration(realmCryptoStoreMigration)
                     .build()
         }
 

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/DefaultCryptoService.kt

@@ -21,13 +21,13 @@ package im.vector.matrix.android.internal.crypto
 import android.content.Context
 import android.os.Handler
 import android.os.Looper
+import androidx.annotation.VisibleForTesting
 import androidx.lifecycle.LiveData
 import com.squareup.moshi.Types
 import com.zhuinden.monarchy.Monarchy
 import dagger.Lazy
 import im.vector.matrix.android.api.MatrixCallback
 import im.vector.matrix.android.api.NoOpMatrixCallback
-import im.vector.matrix.android.api.auth.data.Credentials
 import im.vector.matrix.android.api.crypto.MXCryptoConfig
 import im.vector.matrix.android.api.failure.Failure
 import im.vector.matrix.android.api.listeners.ProgressListener
@@ -45,7 +45,9 @@ import im.vector.matrix.android.api.session.room.model.Membership
 import im.vector.matrix.android.api.session.room.model.RoomHistoryVisibility
 import im.vector.matrix.android.api.session.room.model.RoomHistoryVisibilityContent
 import im.vector.matrix.android.api.session.room.model.RoomMemberSummary
+import im.vector.matrix.android.internal.crypto.actions.EnsureOlmSessionsForDevicesAction
 import im.vector.matrix.android.internal.crypto.actions.MegolmSessionDataImporter
+import im.vector.matrix.android.internal.crypto.actions.MessageEncrypter
 import im.vector.matrix.android.internal.crypto.actions.SetDeviceVerificationAction
 import im.vector.matrix.android.internal.crypto.algorithms.IMXEncrypting
 import im.vector.matrix.android.internal.crypto.algorithms.megolm.MXMegolmEncryptionFactory
@@ -59,6 +61,7 @@ import im.vector.matrix.android.internal.crypto.model.MXDeviceInfo
 import im.vector.matrix.android.internal.crypto.model.MXEncryptEventContentResult
 import im.vector.matrix.android.internal.crypto.model.MXUsersDevicesMap
 import im.vector.matrix.android.internal.crypto.model.event.EncryptedEventContent
+import im.vector.matrix.android.internal.crypto.model.event.OlmEventContent
 import im.vector.matrix.android.internal.crypto.model.event.RoomKeyContent
 import im.vector.matrix.android.internal.crypto.model.event.SecretSendEventContent
 import im.vector.matrix.android.internal.crypto.model.rest.DeviceInfo
@@ -72,13 +75,16 @@ import im.vector.matrix.android.internal.crypto.tasks.DeleteDeviceTask
 import im.vector.matrix.android.internal.crypto.tasks.DeleteDeviceWithUserPasswordTask
 import im.vector.matrix.android.internal.crypto.tasks.GetDeviceInfoTask
 import im.vector.matrix.android.internal.crypto.tasks.GetDevicesTask
+import im.vector.matrix.android.internal.crypto.tasks.SendToDeviceTask
 import im.vector.matrix.android.internal.crypto.tasks.SetDeviceNameTask
 import im.vector.matrix.android.internal.crypto.tasks.UploadKeysTask
 import im.vector.matrix.android.internal.crypto.verification.DefaultVerificationService
 import im.vector.matrix.android.internal.database.model.EventEntity
 import im.vector.matrix.android.internal.database.model.EventEntityFields
 import im.vector.matrix.android.internal.database.query.whereType
+import im.vector.matrix.android.internal.di.DeviceId
 import im.vector.matrix.android.internal.di.MoshiProvider
+import im.vector.matrix.android.internal.di.UserId
 import im.vector.matrix.android.internal.extensions.foldToCallback
 import im.vector.matrix.android.internal.session.SessionScope
 import im.vector.matrix.android.internal.session.room.membership.LoadRoomMembersTask
@@ -116,12 +122,15 @@ import kotlin.math.max
 internal class DefaultCryptoService @Inject constructor(
         // Olm Manager
         private val olmManager: OlmManager,
-        // The credentials,
-        private val credentials: Credentials,
+        @UserId
+        private val userId: String,
+        @DeviceId
+        private val deviceId: String?,
         private val myDeviceInfoHolder: Lazy<MyDeviceInfoHolder>,
         // the crypto store
         private val cryptoStore: IMXCryptoStore,
-
+        // Room encryptors store
+        private val roomEncryptorsStore: RoomEncryptorsStore,
         // Olm device
         private val olmDevice: MXOlmDevice,
         // Set of parameters used to configure/customize the end-to-end crypto.
@@ -162,7 +171,10 @@ internal class DefaultCryptoService @Inject constructor(
         private val monarchy: Monarchy,
         private val coroutineDispatchers: MatrixCoroutineDispatchers,
         private val taskExecutor: TaskExecutor,
-        private val cryptoCoroutineScope: CoroutineScope
+        private val cryptoCoroutineScope: CoroutineScope,
+        private val ensureOlmSessionsForDevicesAction: EnsureOlmSessionsForDevicesAction,
+        private val sendToDeviceTask: SendToDeviceTask,
+        private val messageEncrypter: MessageEncrypter
 ) : CryptoService {
 
     init {
@@ -171,11 +183,13 @@ internal class DefaultCryptoService @Inject constructor(
 
     private val uiHandler = Handler(Looper.getMainLooper())
 
-    // MXEncrypting instance for each room.
-    private val roomEncryptors: MutableMap<String, IMXEncrypting> = HashMap()
     private val isStarting = AtomicBoolean(false)
     private val isStarted = AtomicBoolean(false)
 
+    // The date of the last time we forced establishment
+    // of a new session for each user:device.
+    private val lastNewSessionForcedDates = MXUsersDevicesMap<Long>()
+
     fun onStateEvent(roomId: String, event: Event) {
         when {
             event.getClearType() == EventType.STATE_ROOM_ENCRYPTION         -> onRoomEncryptionEvent(roomId, event)
@@ -199,7 +213,7 @@ internal class DefaultCryptoService @Inject constructor(
                     this.callback = object : MatrixCallback<Unit> {
                         override fun onSuccess(data: Unit) {
                             // bg refresh of crypto device
-                            downloadKeys(listOf(credentials.userId), true, NoOpMatrixCallback())
+                            downloadKeys(listOf(userId), true, NoOpMatrixCallback())
                             callback.onSuccess(data)
                         }
 
@@ -237,15 +251,33 @@ internal class DefaultCryptoService @Inject constructor(
         return myDeviceInfoHolder.get().myDevice
     }
 
-    override fun getDevicesList(callback: MatrixCallback<DevicesListResponse>) {
+    override fun fetchDevicesList(callback: MatrixCallback<DevicesListResponse>) {
         getDevicesTask
                 .configureWith {
                     //                    this.executionThread = TaskThread.CRYPTO
-                    this.callback = callback
+                    this.callback = object : MatrixCallback<DevicesListResponse> {
+                        override fun onFailure(failure: Throwable) {
+                            callback.onFailure(failure)
+                        }
+
+                        override fun onSuccess(data: DevicesListResponse) {
+                            // Save in local DB
+                            cryptoStore.saveMyDevicesInfo(data.devices ?: emptyList())
+                            callback.onSuccess(data)
+                        }
+                    }
                 }
                 .executeBy(taskExecutor)
     }
 
+    override fun getLiveMyDevicesInfo(): LiveData<List<DeviceInfo>> {
+        return cryptoStore.getLiveMyDevicesInfo()
+    }
+
+    override fun getMyDevicesInfo(): List<DeviceInfo> {
+        return cryptoStore.getMyDevicesInfo()
+    }
+
     override fun getDeviceInfo(deviceId: String, callback: MatrixCallback<DeviceInfo>) {
         getDeviceInfoTask
                 .configureWith(GetDeviceInfoTask.Params(deviceId)) {
@@ -304,6 +336,8 @@ internal class DefaultCryptoService @Inject constructor(
         cryptoCoroutineScope.launch(coroutineDispatchers.crypto) {
             internalStart(isInitialSync)
         }
+        // Just update
+        fetchDevicesList(NoOpMatrixCallback())
     }
 
     private suspend fun internalStart(isInitialSync: Boolean) {
@@ -398,7 +432,7 @@ internal class DefaultCryptoService @Inject constructor(
     }
 
     /**
-     * Provides the device information for a device id and a user Id
+     * Provides the device information for a user id and a device Id
      *
      * @param userId   the user id
      * @param deviceId the device id
@@ -412,7 +446,7 @@ internal class DefaultCryptoService @Inject constructor(
     }
 
     override fun getCryptoDeviceInfo(userId: String): List<CryptoDeviceInfo> {
-        return cryptoStore.getUserDevices(userId)?.map { it.value }?.sortedBy { it.deviceId } ?: emptyList()
+        return cryptoStore.getUserDeviceList(userId) ?: emptyList()
     }
 
     override fun getLiveCryptoDeviceInfo(): LiveData<List<CryptoDeviceInfo>> {
@@ -493,14 +527,14 @@ internal class DefaultCryptoService @Inject constructor(
         val existingAlgorithm = cryptoStore.getRoomAlgorithm(roomId)
 
         if (!existingAlgorithm.isNullOrEmpty() && existingAlgorithm != algorithm) {
-            Timber.e("## setEncryptionInRoom() : Ignoring m.room.encryption event which requests a change of config in $roomId")
+            Timber.e("## CRYPTO | setEncryptionInRoom() : Ignoring m.room.encryption event which requests a change of config in $roomId")
             return false
         }
 
         val encryptingClass = MXCryptoAlgorithms.hasEncryptorClassForAlgorithm(algorithm)
 
         if (!encryptingClass) {
-            Timber.e("## setEncryptionInRoom() : Unable to encrypt room $roomId with $algorithm")
+            Timber.e("## CRYPTO | setEncryptionInRoom() : Unable to encrypt room $roomId with $algorithm")
             return false
         }
 
@@ -511,9 +545,7 @@ internal class DefaultCryptoService @Inject constructor(
             else                      -> olmEncryptionFactory.create(roomId)
         }
 
-        synchronized(roomEncryptors) {
-            roomEncryptors.put(roomId, alg)
-        }
+        roomEncryptorsStore.put(roomId, alg)
 
         // if encryption was not previously enabled in this room, we will have been
         // ignoring new device events for these users so far. We may well have
@@ -591,42 +623,44 @@ internal class DefaultCryptoService @Inject constructor(
                                      callback: MatrixCallback<MXEncryptEventContentResult>) {
         cryptoCoroutineScope.launch(coroutineDispatchers.crypto) {
             if (!isStarted()) {
-                Timber.v("## encryptEventContent() : wait after e2e init")
+                Timber.v("## CRYPTO | encryptEventContent() : wait after e2e init")
                 internalStart(false)
             }
             val userIds = getRoomUserIds(roomId)
-            var alg = synchronized(roomEncryptors) {
-                roomEncryptors[roomId]
-            }
+            var alg = roomEncryptorsStore.get(roomId)
             if (alg == null) {
                 val algorithm = getEncryptionAlgorithm(roomId)
                 if (algorithm != null) {
                     if (setEncryptionInRoom(roomId, algorithm, false, userIds)) {
-                        synchronized(roomEncryptors) {
-                            alg = roomEncryptors[roomId]
-                        }
+                        alg = roomEncryptorsStore.get(roomId)
                     }
                 }
             }
             val safeAlgorithm = alg
             if (safeAlgorithm != null) {
                 val t0 = System.currentTimeMillis()
-                Timber.v("## encryptEventContent() starts")
+                Timber.v("## CRYPTO | encryptEventContent() starts")
                 runCatching {
                     val content = safeAlgorithm.encryptEventContent(eventContent, eventType, userIds)
-                    Timber.v("## encryptEventContent() : succeeds after ${System.currentTimeMillis() - t0} ms")
+                    Timber.v("## CRYPTO | encryptEventContent() : succeeds after ${System.currentTimeMillis() - t0} ms")
                     MXEncryptEventContentResult(content, EventType.ENCRYPTED)
                 }.foldToCallback(callback)
             } else {
                 val algorithm = getEncryptionAlgorithm(roomId)
                 val reason = String.format(MXCryptoError.UNABLE_TO_ENCRYPT_REASON,
                         algorithm ?: MXCryptoError.NO_MORE_ALGORITHM_REASON)
-                Timber.e("## encryptEventContent() : $reason")
+                Timber.e("## CRYPTO | encryptEventContent() : $reason")
                 callback.onFailure(Failure.CryptoError(MXCryptoError.Base(MXCryptoError.ErrorType.UNABLE_TO_ENCRYPT, reason)))
             }
         }
     }
 
+    override fun discardOutboundSession(roomId: String) {
+        cryptoCoroutineScope.launch(coroutineDispatchers.crypto) {
+            roomEncryptorsStore.get(roomId)?.discardSessionKey()
+        }
+    }
+
     /**
      * Decrypt an event
      *
@@ -664,20 +698,42 @@ internal class DefaultCryptoService @Inject constructor(
      * @param timeline the id of the timeline where the event is decrypted. It is used to prevent replay attack.
      * @return the MXEventDecryptionResult data, or null in case of error
      */
+    @Throws(MXCryptoError::class)
     private fun internalDecryptEvent(event: Event, timeline: String): MXEventDecryptionResult {
         val eventContent = event.content
         if (eventContent == null) {
-            Timber.e("## decryptEvent : empty event content")
+            Timber.e("## CRYPTO | decryptEvent : empty event content")
             throw MXCryptoError.Base(MXCryptoError.ErrorType.BAD_ENCRYPTED_MESSAGE, MXCryptoError.BAD_ENCRYPTED_MESSAGE_REASON)
         } else {
             val algorithm = eventContent["algorithm"]?.toString()
             val alg = roomDecryptorProvider.getOrCreateRoomDecryptor(event.roomId, algorithm)
             if (alg == null) {
                 val reason = String.format(MXCryptoError.UNABLE_TO_DECRYPT_REASON, event.eventId, algorithm)
-                Timber.e("## decryptEvent() : $reason")
+                Timber.e("## CRYPTO | decryptEvent() : $reason")
                 throw MXCryptoError.Base(MXCryptoError.ErrorType.UNABLE_TO_DECRYPT, reason)
             } else {
-                return alg.decryptEvent(event, timeline)
+                try {
+                    return alg.decryptEvent(event, timeline)
+                } catch (mxCryptoError: MXCryptoError) {
+                    Timber.d("## CRYPTO | internalDecryptEvent : Failed to decrypt ${event.eventId} reason: $mxCryptoError")
+                    if (algorithm == MXCRYPTO_ALGORITHM_OLM) {
+                        if (mxCryptoError is MXCryptoError.Base
+                                && mxCryptoError.errorType == MXCryptoError.ErrorType.BAD_ENCRYPTED_MESSAGE) {
+                            // need to find sending device
+                            val olmContent = event.content.toModel<OlmEventContent>()
+                            cryptoStore.getUserDevices(event.senderId ?: "")
+                                    ?.values
+                                    ?.firstOrNull { it.identityKey() == olmContent?.senderKey }
+                                    ?.let {
+                                        markOlmSessionForUnwedging(event.senderId ?: "", it)
+                                    }
+                                    ?: run {
+                                        Timber.v("## CRYPTO | markOlmSessionForUnwedging() : Failed to find sender crypto device")
+                                    }
+                        }
+                    }
+                    throw mxCryptoError
+                }
             }
         }
     }
@@ -730,30 +786,30 @@ internal class DefaultCryptoService @Inject constructor(
      */
     private fun onRoomKeyEvent(event: Event) {
         val roomKeyContent = event.getClearContent().toModel<RoomKeyContent>() ?: return
-        Timber.v("## GOSSIP onRoomKeyEvent() : type<${event.type}> , sessionId<${roomKeyContent.sessionId}>")
+        Timber.v("## CRYPTO | GOSSIP onRoomKeyEvent() : type<${event.type}> , sessionId<${roomKeyContent.sessionId}>")
         if (roomKeyContent.roomId.isNullOrEmpty() || roomKeyContent.algorithm.isNullOrEmpty()) {
-            Timber.e("## GOSSIP onRoomKeyEvent() : missing fields")
+            Timber.e("## CRYPTO | GOSSIP onRoomKeyEvent() : missing fields")
             return
         }
         val alg = roomDecryptorProvider.getOrCreateRoomDecryptor(roomKeyContent.roomId, roomKeyContent.algorithm)
         if (alg == null) {
-            Timber.e("## GOSSIP onRoomKeyEvent() : Unable to handle keys for ${roomKeyContent.algorithm}")
+            Timber.e("## CRYPTO | GOSSIP onRoomKeyEvent() : Unable to handle keys for ${roomKeyContent.algorithm}")
             return
         }
         alg.onRoomKeyEvent(event, keysBackupService)
     }
 
     private fun onSecretSendReceived(event: Event) {
-        Timber.i("## GOSSIP onSecretSend() : onSecretSendReceived ${event.content?.get("sender_key")}")
+        Timber.i("## CRYPTO | GOSSIP onSecretSend() : onSecretSendReceived ${event.content?.get("sender_key")}")
         if (!event.isEncrypted()) {
             // secret send messages must be encrypted
-            Timber.e("## GOSSIP onSecretSend() :Received unencrypted secret send event")
+            Timber.e("## CRYPTO | GOSSIP onSecretSend() :Received unencrypted secret send event")
             return
         }
 
         // Was that sent by us?
-        if (event.senderId != credentials.userId) {
-            Timber.e("## GOSSIP onSecretSend() : Ignore secret from other user ${event.senderId}")
+        if (event.senderId != userId) {
+            Timber.e("## CRYPTO | GOSSIP onSecretSend() : Ignore secret from other user ${event.senderId}")
             return
         }
 
@@ -763,13 +819,13 @@ internal class DefaultCryptoService @Inject constructor(
                 .getOutgoingSecretKeyRequests().firstOrNull { it.requestId == secretContent.requestId }
 
         if (existingRequest == null) {
-            Timber.i("## GOSSIP onSecretSend() : Ignore secret that was not requested: ${secretContent.requestId}")
+            Timber.i("## CRYPTO | GOSSIP onSecretSend() : Ignore secret that was not requested: ${secretContent.requestId}")
             return
         }
 
         if (!handleSDKLevelGossip(existingRequest.secretName, secretContent.secretValue)) {
             // TODO Ask to application layer?
-            Timber.v("## onSecretSend() : secret not handled by SDK")
+            Timber.v("## CRYPTO | onSecretSend() : secret not handled by SDK")
         }
     }
 
@@ -805,7 +861,7 @@ internal class DefaultCryptoService @Inject constructor(
             try {
                 loadRoomMembersTask.execute(params)
             } catch (throwable: Throwable) {
-                Timber.e(throwable, "## onRoomEncryptionEvent ERROR FAILED TO SETUP CRYPTO ")
+                Timber.e(throwable, "## CRYPTO | onRoomEncryptionEvent ERROR FAILED TO SETUP CRYPTO ")
             } finally {
                 val userIds = getRoomUserIds(roomId)
                 setEncryptionInRoom(roomId, event.content?.get("algorithm")?.toString(), true, userIds)
@@ -835,16 +891,8 @@ internal class DefaultCryptoService @Inject constructor(
      * @param event the membership event causing the change
      */
     private fun onRoomMembershipEvent(roomId: String, event: Event) {
-        val alg: IMXEncrypting?
+        roomEncryptorsStore.get(roomId) ?: /* No encrypting in this room */ return
 
-        synchronized(roomEncryptors) {
-            alg = roomEncryptors[roomId]
-        }
-
-        if (null == alg) {
-            // No encrypting in this room
-            return
-        }
         event.stateKey?.let { userId ->
             val roomMember: RoomMemberSummary? = event.content.toModel()
             val membership = roomMember?.membership
@@ -938,13 +986,13 @@ internal class DefaultCryptoService @Inject constructor(
         cryptoCoroutineScope.launch(coroutineDispatchers.main) {
             runCatching {
                 withContext(coroutineDispatchers.crypto) {
-                    Timber.v("## importRoomKeys starts")
+                    Timber.v("## CRYPTO | importRoomKeys starts")
 
                     val t0 = System.currentTimeMillis()
                     val roomKeys = MXMegolmExportEncryption.decryptMegolmKeyFile(roomKeysAsArray, password)
                     val t1 = System.currentTimeMillis()
 
-                    Timber.v("## importRoomKeys : decryptMegolmKeyFile done in ${t1 - t0} ms")
+                    Timber.v("## CRYPTO | importRoomKeys : decryptMegolmKeyFile done in ${t1 - t0} ms")
 
                     val importedSessions = MoshiProvider.providesMoshi()
                             .adapter<List<MegolmSessionData>>(Types.newParameterizedType(List::class.java, MegolmSessionData::class.java))
@@ -952,7 +1000,7 @@ internal class DefaultCryptoService @Inject constructor(
 
                     val t2 = System.currentTimeMillis()
 
-                    Timber.v("## importRoomKeys : JSON parsing ${t2 - t1} ms")
+                    Timber.v("## CRYPTO | importRoomKeys : JSON parsing ${t2 - t1} ms")
 
                     if (importedSessions == null) {
                         throw Exception("Error")
@@ -1087,7 +1135,7 @@ internal class DefaultCryptoService @Inject constructor(
      */
     override fun reRequestRoomKeyForEvent(event: Event) {
         val wireContent = event.content.toModel<EncryptedEventContent>() ?: return Unit.also {
-            Timber.e("## reRequestRoomKeyForEvent Failed to re-request key, null content")
+            Timber.e("## CRYPTO | reRequestRoomKeyForEvent Failed to re-request key, null content")
         }
 
         val requestBody = RoomKeyRequestBody(
@@ -1102,18 +1150,18 @@ internal class DefaultCryptoService @Inject constructor(
 
     override fun requestRoomKeyForEvent(event: Event) {
         val wireContent = event.content.toModel<EncryptedEventContent>() ?: return Unit.also {
-            Timber.e("## requestRoomKeyForEvent Failed to request key, null content eventId: ${event.eventId}")
+            Timber.e("## CRYPTO | requestRoomKeyForEvent Failed to request key, null content eventId: ${event.eventId}")
         }
 
         cryptoCoroutineScope.launch(coroutineDispatchers.crypto) {
             if (!isStarted()) {
-                Timber.v("## requestRoomKeyForEvent() : wait after e2e init")
+                Timber.v("## CRYPTO | requestRoomKeyForEvent() : wait after e2e init")
                 internalStart(false)
             }
             roomDecryptorProvider
                     .getOrCreateRoomDecryptor(event.roomId, wireContent.algorithm)
                     ?.requestKeysForEvent(event) ?: run {
-                Timber.v("## requestRoomKeyForEvent() : No room decryptor for roomId:${event.roomId} algorithm:${wireContent.algorithm}")
+                Timber.v("## CRYPTO | requestRoomKeyForEvent() : No room decryptor for roomId:${event.roomId} algorithm:${wireContent.algorithm}")
             }
         }
     }
@@ -1136,6 +1184,39 @@ internal class DefaultCryptoService @Inject constructor(
         incomingGossipingRequestManager.removeRoomKeysRequestListener(listener)
     }
 
+    private fun markOlmSessionForUnwedging(senderId: String, deviceInfo: CryptoDeviceInfo) {
+        val deviceKey = deviceInfo.identityKey()
+
+        val lastForcedDate = lastNewSessionForcedDates.getObject(senderId, deviceKey) ?: 0
+        val now = System.currentTimeMillis()
+        if (now - lastForcedDate < CRYPTO_MIN_FORCE_SESSION_PERIOD_MILLIS) {
+            Timber.d("## CRYPTO | markOlmSessionForUnwedging: New session already forced with device at $lastForcedDate. Not forcing another")
+            return
+        }
+
+        Timber.d("## CRYPTO | markOlmSessionForUnwedging from $senderId:${deviceInfo.deviceId}")
+        lastNewSessionForcedDates.setObject(senderId, deviceKey, now)
+
+        cryptoCoroutineScope.launch(coroutineDispatchers.crypto) {
+            ensureOlmSessionsForDevicesAction.handle(mapOf(senderId to listOf(deviceInfo)), force = true)
+
+            // Now send a blank message on that session so the other side knows about it.
+            // (The keyshare request is sent in the clear so that won't do)
+            // We send this first such that, as long as the toDevice messages arrive in the
+            // same order we sent them, the other end will get this first, set up the new session,
+            // then get the keyshare request and send the key over this new session (because it
+            // is the session it has most recently received a message on).
+            val payloadJson = mapOf<String, Any>("type" to EventType.DUMMY)
+
+            val encodedPayload = messageEncrypter.encryptMessage(payloadJson, listOf(deviceInfo))
+            val sendToDeviceMap = MXUsersDevicesMap<Any>()
+            sendToDeviceMap.setObject(senderId, deviceInfo.deviceId, encodedPayload)
+            Timber.v("## CRYPTO | markOlmSessionForUnwedging() : sending to $senderId:${deviceInfo.deviceId}")
+            val sendToDeviceParams = SendToDeviceTask.Params(EventType.ENCRYPTED, sendToDeviceMap)
+            sendToDeviceTask.execute(sendToDeviceParams)
+        }
+    }
+
     /**
      * Provides the list of unknown devices
      *
@@ -1178,7 +1259,7 @@ internal class DefaultCryptoService @Inject constructor(
  * ========================================================================================== */
 
     override fun toString(): String {
-        return "DefaultCryptoService of " + credentials.userId + " (" + credentials.deviceId + ")"
+        return "DefaultCryptoService of $userId ($deviceId)"
     }
 
     override fun getOutgoingRoomKeyRequest(): List<OutgoingRoomKeyRequest> {
@@ -1192,4 +1273,15 @@ internal class DefaultCryptoService @Inject constructor(
     override fun getGossipingEventsTrail(): List<Event> {
         return cryptoStore.getGossipingEventsTrail()
     }
+
+    /* ==========================================================================================
+     * For test only
+     * ========================================================================================== */
+
+    @VisibleForTesting
+    val cryptoStoreForTesting = cryptoStore
+
+    companion object {
+        const val CRYPTO_MIN_FORCE_SESSION_PERIOD_MILLIS = 3_600_000 // one hour
+    }
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/DeviceListManager.kt

@@ -108,7 +108,7 @@ internal class DeviceListManager @Inject constructor(private val cryptoStore: IM
                     res = !notReadyToRetryHS.contains(userId.substringAfterLast(':'))
                 }
             } catch (e: Exception) {
-                Timber.e(e, "## canRetryKeysDownload() failed")
+                Timber.e(e, "## CRYPTO | canRetryKeysDownload() failed")
             }
         }
 
@@ -137,7 +137,7 @@ internal class DeviceListManager @Inject constructor(private val cryptoStore: IM
 
             for (userId in userIds) {
                 if (!deviceTrackingStatuses.containsKey(userId) || TRACKING_STATUS_NOT_TRACKED == deviceTrackingStatuses[userId]) {
-                    Timber.v("## startTrackingDeviceList() : Now tracking device list for $userId")
+                    Timber.v("## CRYPTO | startTrackingDeviceList() : Now tracking device list for $userId")
                     deviceTrackingStatuses[userId] = TRACKING_STATUS_PENDING_DOWNLOAD
                     isUpdated = true
                 }
@@ -161,7 +161,7 @@ internal class DeviceListManager @Inject constructor(private val cryptoStore: IM
 
         for (userId in changed) {
             if (deviceTrackingStatuses.containsKey(userId)) {
-                Timber.v("## invalidateUserDeviceList() : Marking device list outdated for $userId")
+                Timber.v("## CRYPTO | invalidateUserDeviceList() : Marking device list outdated for $userId")
                 deviceTrackingStatuses[userId] = TRACKING_STATUS_PENDING_DOWNLOAD
                 isUpdated = true
             }
@@ -169,7 +169,7 @@ internal class DeviceListManager @Inject constructor(private val cryptoStore: IM
 
         for (userId in left) {
             if (deviceTrackingStatuses.containsKey(userId)) {
-                Timber.v("## invalidateUserDeviceList() : No longer tracking device list for $userId")
+                Timber.v("## CRYPTO | invalidateUserDeviceList() : No longer tracking device list for $userId")
                 deviceTrackingStatuses[userId] = TRACKING_STATUS_NOT_TRACKED
                 isUpdated = true
             }
@@ -259,7 +259,7 @@ internal class DeviceListManager @Inject constructor(private val cryptoStore: IM
      * @param forceDownload Always download the keys even if cached.
      */
     suspend fun downloadKeys(userIds: List<String>?, forceDownload: Boolean): MXUsersDevicesMap<CryptoDeviceInfo> {
-        Timber.v("## downloadKeys() : forceDownload $forceDownload : $userIds")
+        Timber.v("## CRYPTO | downloadKeys() : forceDownload $forceDownload : $userIds")
         // Map from userId -> deviceId -> MXDeviceInfo
         val stored = MXUsersDevicesMap<CryptoDeviceInfo>()
 
@@ -288,13 +288,13 @@ internal class DeviceListManager @Inject constructor(private val cryptoStore: IM
             }
         }
         return if (downloadUsers.isEmpty()) {
-            Timber.v("## downloadKeys() : no new user device")
+            Timber.v("## CRYPTO | downloadKeys() : no new user device")
             stored
         } else {
-            Timber.v("## downloadKeys() : starts")
+            Timber.v("## CRYPTO | downloadKeys() : starts")
             val t0 = System.currentTimeMillis()
             val result = doKeyDownloadForUsers(downloadUsers)
-            Timber.v("## downloadKeys() : doKeyDownloadForUsers succeeds after ${System.currentTimeMillis() - t0} ms")
+            Timber.v("## CRYPTO | downloadKeys() : doKeyDownloadForUsers succeeds after ${System.currentTimeMillis() - t0} ms")
             result.also {
                 it.addEntriesFromMap(stored)
             }
@@ -307,7 +307,7 @@ internal class DeviceListManager @Inject constructor(private val cryptoStore: IM
      * @param downloadUsers the user ids list
      */
     private suspend fun doKeyDownloadForUsers(downloadUsers: List<String>): MXUsersDevicesMap<CryptoDeviceInfo> {
-        Timber.v("## doKeyDownloadForUsers() : doKeyDownloadForUsers $downloadUsers")
+        Timber.v("## CRYPTO | doKeyDownloadForUsers() : doKeyDownloadForUsers $downloadUsers")
         // get the user ids which did not already trigger a keys download
         val filteredUsers = downloadUsers.filter { MatrixPatterns.isUserId(it) }
         if (filteredUsers.isEmpty()) {
@@ -318,16 +318,16 @@ internal class DeviceListManager @Inject constructor(private val cryptoStore: IM
         val response = try {
             downloadKeysForUsersTask.execute(params)
         } catch (throwable: Throwable) {
-            Timber.e(throwable, "##doKeyDownloadForUsers(): error")
+            Timber.e(throwable, "## CRYPTO | doKeyDownloadForUsers(): error")
             onKeysDownloadFailed(filteredUsers)
             throw throwable
         }
-        Timber.v("## doKeyDownloadForUsers() : Got keys for " + filteredUsers.size + " users")
+        Timber.v("## CRYPTO | doKeyDownloadForUsers() : Got keys for " + filteredUsers.size + " users")
         for (userId in filteredUsers) {
             // al devices =
             val models = response.deviceKeys?.get(userId)?.mapValues { entry -> CryptoInfoMapper.map(entry.value) }
 
-            Timber.v("## doKeyDownloadForUsers() : Got keys for $userId : $models")
+            Timber.v("## CRYPTO | doKeyDownloadForUsers() : Got keys for $userId : $models")
             if (!models.isNullOrEmpty()) {
                 val workingCopy = models.toMutableMap()
                 for ((deviceId, deviceInfo) in models) {
@@ -361,13 +361,13 @@ internal class DeviceListManager @Inject constructor(private val cryptoStore: IM
 
             // Handle cross signing keys update
             val masterKey = response.masterKeys?.get(userId)?.toCryptoModel().also {
-                Timber.v("## CrossSigning : Got keys for $userId : MSK ${it?.unpaddedBase64PublicKey}")
+                Timber.v("## CRYPTO | CrossSigning : Got keys for $userId : MSK ${it?.unpaddedBase64PublicKey}")
             }
             val selfSigningKey = response.selfSigningKeys?.get(userId)?.toCryptoModel()?.also {
-                Timber.v("## CrossSigning : Got keys for $userId : SSK ${it.unpaddedBase64PublicKey}")
+                Timber.v("## CRYPTO | CrossSigning : Got keys for $userId : SSK ${it.unpaddedBase64PublicKey}")
             }
             val userSigningKey = response.userSigningKeys?.get(userId)?.toCryptoModel()?.also {
-                Timber.v("## CrossSigning : Got keys for $userId : USK ${it.unpaddedBase64PublicKey}")
+                Timber.v("## CRYPTO | CrossSigning : Got keys for $userId : USK ${it.unpaddedBase64PublicKey}")
             }
             cryptoStore.storeUserCrossSigningKeys(
                     userId,
@@ -395,28 +395,28 @@ internal class DeviceListManager @Inject constructor(private val cryptoStore: IM
      */
     private fun validateDeviceKeys(deviceKeys: CryptoDeviceInfo?, userId: String, deviceId: String, previouslyStoredDeviceKeys: CryptoDeviceInfo?): Boolean {
         if (null == deviceKeys) {
-            Timber.e("## validateDeviceKeys() : deviceKeys is null from $userId:$deviceId")
+            Timber.e("## CRYPTO | validateDeviceKeys() : deviceKeys is null from $userId:$deviceId")
             return false
         }
 
         if (null == deviceKeys.keys) {
-            Timber.e("## validateDeviceKeys() : deviceKeys.keys is null from $userId:$deviceId")
+            Timber.e("## CRYPTO | validateDeviceKeys() : deviceKeys.keys is null from $userId:$deviceId")
             return false
         }
 
         if (null == deviceKeys.signatures) {
-            Timber.e("## validateDeviceKeys() : deviceKeys.signatures is null from $userId:$deviceId")
+            Timber.e("## CRYPTO | validateDeviceKeys() : deviceKeys.signatures is null from $userId:$deviceId")
             return false
         }
 
         // Check that the user_id and device_id in the received deviceKeys are correct
         if (deviceKeys.userId != userId) {
-            Timber.e("## validateDeviceKeys() : Mismatched user_id ${deviceKeys.userId} from $userId:$deviceId")
+            Timber.e("## CRYPTO | validateDeviceKeys() : Mismatched user_id ${deviceKeys.userId} from $userId:$deviceId")
             return false
         }
 
         if (deviceKeys.deviceId != deviceId) {
-            Timber.e("## validateDeviceKeys() : Mismatched device_id ${deviceKeys.deviceId} from $userId:$deviceId")
+            Timber.e("## CRYPTO | validateDeviceKeys() : Mismatched device_id ${deviceKeys.deviceId} from $userId:$deviceId")
             return false
         }
 
@@ -424,21 +424,21 @@ internal class DeviceListManager @Inject constructor(private val cryptoStore: IM
         val signKey = deviceKeys.keys[signKeyId]
 
         if (null == signKey) {
-            Timber.e("## validateDeviceKeys() : Device $userId:${deviceKeys.deviceId} has no ed25519 key")
+            Timber.e("## CRYPTO | validateDeviceKeys() : Device $userId:${deviceKeys.deviceId} has no ed25519 key")
             return false
         }
 
         val signatureMap = deviceKeys.signatures[userId]
 
         if (null == signatureMap) {
-            Timber.e("## validateDeviceKeys() : Device $userId:${deviceKeys.deviceId} has no map for $userId")
+            Timber.e("## CRYPTO | validateDeviceKeys() : Device $userId:${deviceKeys.deviceId} has no map for $userId")
             return false
         }
 
         val signature = signatureMap[signKeyId]
 
         if (null == signature) {
-            Timber.e("## validateDeviceKeys() : Device $userId:${deviceKeys.deviceId} is not signed")
+            Timber.e("## CRYPTO | validateDeviceKeys() : Device $userId:${deviceKeys.deviceId} is not signed")
             return false
         }
 
@@ -453,7 +453,7 @@ internal class DeviceListManager @Inject constructor(private val cryptoStore: IM
         }
 
         if (!isVerified) {
-            Timber.e("## validateDeviceKeys() : Unable to verify signature on device " + userId + ":"
+            Timber.e("## CRYPTO | validateDeviceKeys() : Unable to verify signature on device " + userId + ":"
                     + deviceKeys.deviceId + " with error " + errorMessage)
             return false
         }
@@ -464,12 +464,12 @@ internal class DeviceListManager @Inject constructor(private val cryptoStore: IM
                 // best off sticking with the original keys.
                 //
                 // Should we warn the user about it somehow?
-                Timber.e("## validateDeviceKeys() : WARNING:Ed25519 key for device " + userId + ":"
+                Timber.e("## CRYPTO | validateDeviceKeys() : WARNING:Ed25519 key for device " + userId + ":"
                         + deviceKeys.deviceId + " has changed : "
                         + previouslyStoredDeviceKeys.fingerprint() + " -> " + signKey)
 
-                Timber.e("## validateDeviceKeys() : $previouslyStoredDeviceKeys -> $deviceKeys")
-                Timber.e("## validateDeviceKeys() : ${previouslyStoredDeviceKeys.keys} -> ${deviceKeys.keys}")
+                Timber.e("## CRYPTO | validateDeviceKeys() : $previouslyStoredDeviceKeys -> $deviceKeys")
+                Timber.e("## CRYPTO | validateDeviceKeys() : ${previouslyStoredDeviceKeys.keys} -> ${deviceKeys.keys}")
 
                 return false
             }
@@ -501,10 +501,10 @@ internal class DeviceListManager @Inject constructor(private val cryptoStore: IM
             doKeyDownloadForUsers(users)
         }.fold(
                 {
-                    Timber.v("## refreshOutdatedDeviceLists() : done")
+                    Timber.v("## CRYPTO | refreshOutdatedDeviceLists() : done")
                 },
                 {
-                    Timber.e(it, "## refreshOutdatedDeviceLists() : ERROR updating device keys for users $users")
+                    Timber.e(it, "## CRYPTO | refreshOutdatedDeviceLists() : ERROR updating device keys for users $users")
                 }
         )
     }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/IncomingGossipingRequestManager.kt

@@ -32,7 +32,10 @@ import im.vector.matrix.android.internal.crypto.model.rest.GossipingToDeviceObje
 import im.vector.matrix.android.internal.crypto.store.IMXCryptoStore
 import im.vector.matrix.android.internal.di.SessionId
 import im.vector.matrix.android.internal.session.SessionScope
+import im.vector.matrix.android.internal.util.MatrixCoroutineDispatchers
 import im.vector.matrix.android.internal.worker.WorkerParamsFactory
+import kotlinx.coroutines.CoroutineScope
+import kotlinx.coroutines.launch
 import timber.log.Timber
 import javax.inject.Inject
 
@@ -43,7 +46,10 @@ internal class IncomingGossipingRequestManager @Inject constructor(
         private val cryptoStore: IMXCryptoStore,
         private val cryptoConfig: MXCryptoConfig,
         private val gossipingWorkManager: GossipingWorkManager,
-        private val roomDecryptorProvider: RoomDecryptorProvider) {
+        private val roomEncryptorsStore: RoomEncryptorsStore,
+        private val roomDecryptorProvider: RoomDecryptorProvider,
+        private val coroutineDispatchers: MatrixCoroutineDispatchers,
+        private val cryptoCoroutineScope: CoroutineScope) {
 
     // list of IncomingRoomKeyRequests/IncomingRoomKeyRequestCancellations
     // we received in the current sync.
@@ -90,7 +96,7 @@ internal class IncomingGossipingRequestManager @Inject constructor(
      * @param event the announcement event.
      */
     fun onGossipingRequestEvent(event: Event) {
-        Timber.v("## GOSSIP onGossipingRequestEvent type ${event.type} from user ${event.senderId}")
+        Timber.v("## CRYPTO | GOSSIP onGossipingRequestEvent type ${event.type} from user ${event.senderId}")
         val roomKeyShare = event.getClearContent().toModel<GossipingDefaultContent>()
         val ageLocalTs = event.unsignedData?.age?.let { System.currentTimeMillis() - it }
         when (roomKeyShare?.action) {
@@ -155,7 +161,7 @@ internal class IncomingGossipingRequestManager @Inject constructor(
         }
 
         receivedRequestCancellations?.forEach { request ->
-            Timber.v("## GOSSIP processReceivedGossipingRequests() : m.room_key_request cancellation $request")
+            Timber.v("## CRYPTO | GOSSIP processReceivedGossipingRequests() : m.room_key_request cancellation $request")
             // we should probably only notify the app of cancellations we told it
             // about, but we don't currently have a record of that, so we just pass
             // everything through.
@@ -178,17 +184,42 @@ internal class IncomingGossipingRequestManager @Inject constructor(
     }
 
     private fun processIncomingRoomKeyRequest(request: IncomingRoomKeyRequest) {
-        val userId = request.userId
-        val deviceId = request.deviceId
-        val body = request.requestBody
-        val roomId = body!!.roomId
-        val alg = body.algorithm
+        val userId = request.userId ?: return
+        val deviceId = request.deviceId ?: return
+        val body = request.requestBody ?: return
+        val roomId = body.roomId ?: return
+        val alg = body.algorithm ?: return
 
-        Timber.v("## GOSSIP processIncomingRoomKeyRequest from $userId:$deviceId for $roomId / ${body.sessionId} id ${request.requestId}")
-        if (userId == null || credentials.userId != userId) {
-            // TODO: determine if we sent this device the keys already: in
-            Timber.w("## GOSSIP processReceivedGossipingRequests() : Ignoring room key request from other user for now")
-            cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED)
+        Timber.v("## CRYPTO | GOSSIP processIncomingRoomKeyRequest from $userId:$deviceId for $roomId / ${body.sessionId} id ${request.requestId}")
+        if (credentials.userId != userId) {
+            Timber.w("## CRYPTO | GOSSIP processReceivedGossipingRequests() : room key request from other user")
+            val senderKey = body.senderKey ?: return Unit
+                    .also { Timber.w("missing senderKey") }
+                    .also { cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED) }
+            val sessionId = body.sessionId ?: return Unit
+                    .also { Timber.w("missing sessionId") }
+                    .also { cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED) }
+
+            if (alg != MXCRYPTO_ALGORITHM_MEGOLM) {
+                return Unit
+                        .also { Timber.w("Only megolm is accepted here") }
+                        .also { cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED) }
+            }
+
+            val roomEncryptor = roomEncryptorsStore.get(roomId) ?: return Unit
+                    .also { Timber.w("no room Encryptor") }
+                    .also { cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED) }
+
+            cryptoCoroutineScope.launch(coroutineDispatchers.crypto) {
+                val isSuccess = roomEncryptor.reshareKey(sessionId, userId, deviceId, senderKey)
+
+                if (isSuccess) {
+                    cryptoStore.updateGossipingRequestState(request, GossipingRequestState.ACCEPTED)
+                } else {
+                    cryptoStore.updateGossipingRequestState(request, GossipingRequestState.UNABLE_TO_PROCESS)
+                }
+            }
+            cryptoStore.updateGossipingRequestState(request, GossipingRequestState.RE_REQUESTED)
             return
         }
         // TODO: should we queue up requests we don't yet have keys for, in case they turn up later?
@@ -196,18 +227,18 @@ internal class IncomingGossipingRequestManager @Inject constructor(
         // the keys for the requested events, and can drop the requests.
         val decryptor = roomDecryptorProvider.getRoomDecryptor(roomId, alg)
         if (null == decryptor) {
-            Timber.w("## GOSSIP processReceivedGossipingRequests() : room key request for unknown $alg in room $roomId")
+            Timber.w("## CRYPTO | GOSSIP processReceivedGossipingRequests() : room key request for unknown $alg in room $roomId")
             cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED)
             return
         }
         if (!decryptor.hasKeysForKeyRequest(request)) {
-            Timber.w("## GOSSIP processReceivedGossipingRequests() : room key request for unknown session ${body.sessionId!!}")
+            Timber.w("## CRYPTO | GOSSIP processReceivedGossipingRequests() : room key request for unknown session ${body.sessionId!!}")
             cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED)
             return
         }
 
         if (credentials.deviceId == deviceId && credentials.userId == userId) {
-            Timber.v("## GOSSIP processReceivedGossipingRequests() : oneself device - ignored")
+            Timber.v("## CRYPTO | GOSSIP processReceivedGossipingRequests() : oneself device - ignored")
             cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED)
             return
         }
@@ -219,16 +250,16 @@ internal class IncomingGossipingRequestManager @Inject constructor(
             cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED)
         }
         // if the device is verified already, share the keys
-        val device = cryptoStore.getUserDevice(userId, deviceId!!)
+        val device = cryptoStore.getUserDevice(userId, deviceId)
         if (device != null) {
             if (device.isVerified) {
-                Timber.v("## GOSSIP processReceivedGossipingRequests() : device is already verified: sharing keys")
+                Timber.v("## CRYPTO | GOSSIP processReceivedGossipingRequests() : device is already verified: sharing keys")
                 request.share?.run()
                 return
             }
 
             if (device.isBlocked) {
-                Timber.v("## GOSSIP processReceivedGossipingRequests() : device is blocked -> ignored")
+                Timber.v("## CRYPTO | GOSSIP processReceivedGossipingRequests() : device is blocked -> ignored")
                 cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED)
                 return
             }
@@ -236,7 +267,7 @@ internal class IncomingGossipingRequestManager @Inject constructor(
 
         // As per config we automatically discard untrusted devices request
         if (cryptoConfig.discardRoomKeyRequestsFromUntrustedDevices) {
-            Timber.v("## processReceivedGossipingRequests() : discardRoomKeyRequestsFromUntrustedDevices")
+            Timber.v("## CRYPTO | processReceivedGossipingRequests() : discardRoomKeyRequestsFromUntrustedDevices")
             // At this point the device is unknown, we don't want to bother user with that
             cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED)
             return
@@ -249,30 +280,30 @@ internal class IncomingGossipingRequestManager @Inject constructor(
     private fun processIncomingSecretShareRequest(request: IncomingSecretShareRequest) {
         val secretName = request.secretName ?: return Unit.also {
             cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED)
-            Timber.v("## GOSSIP processIncomingSecretShareRequest() : Missing secret name")
+            Timber.v("## CRYPTO | GOSSIP processIncomingSecretShareRequest() : Missing secret name")
         }
 
         val userId = request.userId
         if (userId == null || credentials.userId != userId) {
-            Timber.e("## GOSSIP processIncomingSecretShareRequest() : Ignoring secret share request from other users")
+            Timber.e("## CRYPTO | GOSSIP processIncomingSecretShareRequest() : Ignoring secret share request from other users")
             cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED)
             return
         }
 
         val deviceId = request.deviceId
                 ?: return Unit.also {
-                    Timber.e("## GOSSIP processIncomingSecretShareRequest() : Malformed request, no ")
+                    Timber.e("## CRYPTO | GOSSIP processIncomingSecretShareRequest() : Malformed request, no ")
                     cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED)
                 }
 
         val device = cryptoStore.getUserDevice(userId, deviceId)
                 ?: return Unit.also {
-                    Timber.e("## GOSSIP processIncomingSecretShareRequest() : Received secret share request from unknown device ${request.deviceId}")
+                    Timber.e("## CRYPTO | GOSSIP processIncomingSecretShareRequest() : Received secret share request from unknown device ${request.deviceId}")
                     cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED)
                 }
 
         if (!device.isVerified || device.isBlocked) {
-            Timber.v("## GOSSIP processIncomingSecretShareRequest() : Ignoring secret share request from untrusted/blocked session $device")
+            Timber.v("## CRYPTO | GOSSIP processIncomingSecretShareRequest() : Ignoring secret share request from untrusted/blocked session $device")
             cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED)
             return
         }
@@ -289,7 +320,7 @@ internal class IncomingGossipingRequestManager @Inject constructor(
                     }
             else                       -> null
         }?.let { secretValue ->
-            Timber.i("## GOSSIP processIncomingSecretShareRequest() : Sharing secret $secretName with $device locally trusted")
+            Timber.i("## CRYPTO | GOSSIP processIncomingSecretShareRequest() : Sharing secret $secretName with $device locally trusted")
             if (isDeviceLocallyVerified == true && hasBeenVerifiedLessThanFiveMinutesFromNow(deviceId)) {
                 val params = SendGossipWorker.Params(
                         sessionId = sessionId,
@@ -301,13 +332,13 @@ internal class IncomingGossipingRequestManager @Inject constructor(
                 val workRequest = gossipingWorkManager.createWork<SendGossipWorker>(WorkerParamsFactory.toData(params), true)
                 gossipingWorkManager.postWork(workRequest)
             } else {
-                Timber.v("## GOSSIP processIncomingSecretShareRequest() : Can't share secret $secretName with $device, verification too old")
+                Timber.v("## CRYPTO | GOSSIP processIncomingSecretShareRequest() : Can't share secret $secretName with $device, verification too old")
                 cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED)
             }
             return
         }
 
-        Timber.v("## GOSSIP processIncomingSecretShareRequest() : $secretName unknown at SDK level, asking to app layer")
+        Timber.v("## CRYPTO | GOSSIP processIncomingSecretShareRequest() : $secretName unknown at SDK level, asking to app layer")
 
         request.ignore = Runnable {
             cryptoStore.updateGossipingRequestState(request, GossipingRequestState.REJECTED)
@@ -341,7 +372,7 @@ internal class IncomingGossipingRequestManager @Inject constructor(
                 try {
                     listener.onRoomKeyRequest(request)
                 } catch (e: Exception) {
-                    Timber.e(e, "## onRoomKeyRequest() failed")
+                    Timber.e(e, "## CRYPTO | onRoomKeyRequest() failed")
                 }
             }
         }
@@ -358,7 +389,7 @@ internal class IncomingGossipingRequestManager @Inject constructor(
                         return
                     }
                 } catch (e: Exception) {
-                    Timber.e(e, "## GOSSIP onRoomKeyRequest() failed")
+                    Timber.e(e, "## CRYPTO | GOSSIP onRoomKeyRequest() failed")
                 }
             }
         }
@@ -377,7 +408,7 @@ internal class IncomingGossipingRequestManager @Inject constructor(
                 try {
                     listener.onRoomKeyRequestCancellation(request)
                 } catch (e: Exception) {
-                    Timber.e(e, "## GOSSIP onRoomKeyRequestCancellation() failed")
+                    Timber.e(e, "## CRYPTO | GOSSIP onRoomKeyRequestCancellation() failed")
                 }
             }
         }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/MXOlmDevice.kt

@@ -21,7 +21,7 @@ import im.vector.matrix.android.api.session.crypto.MXCryptoError
 import im.vector.matrix.android.api.util.JSON_DICT_PARAMETERIZED_TYPE
 import im.vector.matrix.android.api.util.JsonDict
 import im.vector.matrix.android.internal.crypto.algorithms.olm.OlmDecryptionResult
-import im.vector.matrix.android.internal.crypto.model.OlmInboundGroupSessionWrapper
+import im.vector.matrix.android.internal.crypto.model.OlmInboundGroupSessionWrapper2
 import im.vector.matrix.android.internal.crypto.model.OlmSessionWrapper
 import im.vector.matrix.android.internal.crypto.store.IMXCryptoStore
 import im.vector.matrix.android.internal.di.MoshiProvider
@@ -342,6 +342,8 @@ internal class MXOlmDevice @Inject constructor(
             } catch (e: Exception) {
                 Timber.e(e, "## encryptMessage() : failed")
             }
+        } else {
+            Timber.e("## encryptMessage() : Failed to encrypt unknown session $sessionId")
         }
 
         return res
@@ -486,7 +488,7 @@ internal class MXOlmDevice @Inject constructor(
                                forwardingCurve25519KeyChain: List<String>,
                                keysClaimed: Map<String, String>,
                                exportFormat: Boolean): Boolean {
-        val session = OlmInboundGroupSessionWrapper(sessionKey, exportFormat)
+        val session = OlmInboundGroupSessionWrapper2(sessionKey, exportFormat)
         runCatching { getInboundGroupSession(sessionId, senderKey, roomId) }
                 .fold(
                         {
@@ -541,18 +543,18 @@ internal class MXOlmDevice @Inject constructor(
      * @param megolmSessionsData the megolm sessions data
      * @return the successfully imported sessions.
      */
-    fun importInboundGroupSessions(megolmSessionsData: List<MegolmSessionData>): List<OlmInboundGroupSessionWrapper> {
-        val sessions = ArrayList<OlmInboundGroupSessionWrapper>(megolmSessionsData.size)
+    fun importInboundGroupSessions(megolmSessionsData: List<MegolmSessionData>): List<OlmInboundGroupSessionWrapper2> {
+        val sessions = ArrayList<OlmInboundGroupSessionWrapper2>(megolmSessionsData.size)
 
         for (megolmSessionData in megolmSessionsData) {
             val sessionId = megolmSessionData.sessionId
             val senderKey = megolmSessionData.senderKey
             val roomId = megolmSessionData.roomId
 
-            var session: OlmInboundGroupSessionWrapper? = null
+            var session: OlmInboundGroupSessionWrapper2? = null
 
             try {
-                session = OlmInboundGroupSessionWrapper(megolmSessionData)
+                session = OlmInboundGroupSessionWrapper2(megolmSessionData)
             } catch (e: Exception) {
                 Timber.e(e, "## importInboundGroupSession() : Update for megolm session $senderKey/$sessionId")
             }
@@ -625,6 +627,7 @@ internal class MXOlmDevice @Inject constructor(
      * @param senderKey the base64-encoded curve25519 key of the sender.
      * @return the decrypting result. Nil if the sessionId is unknown.
      */
+    @Throws(MXCryptoError::class)
     fun decryptGroupMessage(body: String,
                             roomId: String,
                             timeline: String?,
@@ -662,8 +665,7 @@ internal class MXOlmDevice @Inject constructor(
                 adapter.fromJson(payloadString)
             } catch (e: Exception) {
                 Timber.e("## decryptGroupMessage() : fails to parse the payload")
-                throw
-                MXCryptoError.Base(MXCryptoError.ErrorType.BAD_DECRYPTED_FORMAT, MXCryptoError.BAD_DECRYPTED_FORMAT_TEXT_REASON)
+                throw MXCryptoError.Base(MXCryptoError.ErrorType.BAD_DECRYPTED_FORMAT, MXCryptoError.BAD_DECRYPTED_FORMAT_TEXT_REASON)
             }
 
             return OlmDecryptionResult(
@@ -739,7 +741,7 @@ internal class MXOlmDevice @Inject constructor(
      * @param senderKey the base64-encoded curve25519 key of the sender.
      * @return the inbound group session.
      */
-    fun getInboundGroupSession(sessionId: String?, senderKey: String?, roomId: String?): OlmInboundGroupSessionWrapper {
+    fun getInboundGroupSession(sessionId: String?, senderKey: String?, roomId: String?): OlmInboundGroupSessionWrapper2 {
         if (sessionId.isNullOrBlank() || senderKey.isNullOrBlank()) {
             throw MXCryptoError.Base(MXCryptoError.ErrorType.MISSING_SENDER_KEY, MXCryptoError.ERROR_MISSING_PROPERTY_REASON)
         }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/OutgoingGossipingRequestManager.kt

@@ -55,7 +55,7 @@ internal class OutgoingGossipingRequestManager @Inject constructor(
             cryptoStore.getOrAddOutgoingRoomKeyRequest(requestBody, recipients)?.let {
                 // Don't resend if it's already done, you need to cancel first (reRequest)
                 if (it.state == OutgoingGossipingRequestState.SENDING || it.state == OutgoingGossipingRequestState.SENT) {
-                    Timber.v("## GOSSIP sendOutgoingRoomKeyRequest() : we already request for that session: $it")
+                    Timber.v("## CRYPTO - GOSSIP sendOutgoingRoomKeyRequest() : we already request for that session: $it")
                     return@launch
                 }
 
@@ -72,7 +72,7 @@ internal class OutgoingGossipingRequestManager @Inject constructor(
             cryptoStore.getOrAddOutgoingSecretShareRequest(secretName, recipients)?.let {
                 // TODO check if there is already one that is being sent?
                 if (it.state == OutgoingGossipingRequestState.SENDING || it.state == OutgoingGossipingRequestState.SENT) {
-                    Timber.v("## GOSSIP sendSecretShareRequest() : we already request for that session: $it")
+                    Timber.v("## CRYPTO - GOSSIP sendSecretShareRequest() : we already request for that session: $it")
                     return@launch
                 }
 
@@ -113,7 +113,7 @@ internal class OutgoingGossipingRequestManager @Inject constructor(
         val req = cryptoStore.getOutgoingRoomKeyRequest(requestBody)
                 ?: // no request was made for this key
                 return Unit.also {
-                    Timber.v("## GOSSIP cancelRoomKeyRequest() Unknown request")
+                    Timber.v("## CRYPTO - GOSSIP cancelRoomKeyRequest() Unknown request $requestBody")
                 }
 
         sendOutgoingRoomKeyRequestCancellation(req, andResend)
@@ -125,7 +125,7 @@ internal class OutgoingGossipingRequestManager @Inject constructor(
      * @param request the request
      */
     private fun sendOutgoingGossipingRequest(request: OutgoingGossipingRequest) {
-        Timber.v("## GOSSIP sendOutgoingRoomKeyRequest() : Requesting keys $request")
+        Timber.v("## CRYPTO - GOSSIP sendOutgoingRoomKeyRequest() : Requesting keys $request")
 
         val params = SendGossipRequestWorker.Params(
                 sessionId = sessionId,
@@ -143,7 +143,7 @@ internal class OutgoingGossipingRequestManager @Inject constructor(
      * @param request the request
      */
     private fun sendOutgoingRoomKeyRequestCancellation(request: OutgoingRoomKeyRequest, resend: Boolean = false) {
-        Timber.v("$request")
+        Timber.v("## CRYPTO - sendOutgoingRoomKeyRequestCancellation $request")
         val params = CancelGossipRequestWorker.Params.fromRequest(sessionId, request)
         cryptoStore.updateOutgoingGossipingRequestState(request.requestId, OutgoingGossipingRequestState.CANCELLING)
 

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/RoomEncryptorsStore.kt

@@ -0,0 +1,40 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package im.vector.matrix.android.internal.crypto
+
+import im.vector.matrix.android.internal.crypto.algorithms.IMXEncrypting
+import im.vector.matrix.android.internal.session.SessionScope
+import javax.inject.Inject
+
+@SessionScope
+internal class RoomEncryptorsStore @Inject constructor() {
+
+    // MXEncrypting instance for each room.
+    private val roomEncryptors = mutableMapOf<String, IMXEncrypting>()
+
+    fun put(roomId: String, alg: IMXEncrypting) {
+        synchronized(roomEncryptors) {
+            roomEncryptors.put(roomId, alg)
+        }
+    }
+
+    fun get(roomId: String): IMXEncrypting? {
+        return synchronized(roomEncryptors) {
+            roomEncryptors[roomId]
+        }
+    }
+}

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/actions/EnsureOlmSessionsForDevicesAction.kt

@@ -25,10 +25,11 @@ import im.vector.matrix.android.internal.crypto.tasks.ClaimOneTimeKeysForUsersDe
 import timber.log.Timber
 import javax.inject.Inject
 
-internal class EnsureOlmSessionsForDevicesAction @Inject constructor(private val olmDevice: MXOlmDevice,
-                                                                     private val oneTimeKeysForUsersDeviceTask: ClaimOneTimeKeysForUsersDeviceTask) {
+internal class EnsureOlmSessionsForDevicesAction @Inject constructor(
+        private val olmDevice: MXOlmDevice,
+        private val oneTimeKeysForUsersDeviceTask: ClaimOneTimeKeysForUsersDeviceTask) {
 
-    suspend fun handle(devicesByUser: Map<String, List<CryptoDeviceInfo>>): MXUsersDevicesMap<MXOlmSessionResult> {
+    suspend fun handle(devicesByUser: Map<String, List<CryptoDeviceInfo>>, force: Boolean = false): MXUsersDevicesMap<MXOlmSessionResult> {
         val devicesWithoutSession = ArrayList<CryptoDeviceInfo>()
 
         val results = MXUsersDevicesMap<MXOlmSessionResult>()
@@ -40,7 +41,7 @@ internal class EnsureOlmSessionsForDevicesAction @Inject constructor(private val
 
                 val sessionId = olmDevice.getSessionId(key!!)
 
-                if (sessionId.isNullOrEmpty()) {
+                if (sessionId.isNullOrEmpty() || force) {
                     devicesWithoutSession.add(deviceInfo)
                 }
 
@@ -68,11 +69,11 @@ internal class EnsureOlmSessionsForDevicesAction @Inject constructor(private val
         //
         // That should eventually resolve itself, but it's poor form.
 
-        Timber.v("## claimOneTimeKeysForUsersDevices() : $usersDevicesToClaim")
+        Timber.v("## CRYPTO | claimOneTimeKeysForUsersDevices() : $usersDevicesToClaim")
 
         val claimParams = ClaimOneTimeKeysForUsersDeviceTask.Params(usersDevicesToClaim)
         val oneTimeKeys = oneTimeKeysForUsersDeviceTask.execute(claimParams)
-        Timber.v("## claimOneTimeKeysForUsersDevices() : keysClaimResponse.oneTimeKeys: $oneTimeKeys")
+        Timber.v("## CRYPTO | claimOneTimeKeysForUsersDevices() : keysClaimResponse.oneTimeKeys: $oneTimeKeys")
         for ((userId, deviceInfos) in devicesByUser) {
             for (deviceInfo in deviceInfos) {
                 var oneTimeKey: MXKey? = null
@@ -80,7 +81,7 @@ internal class EnsureOlmSessionsForDevicesAction @Inject constructor(private val
                 if (null != deviceIds) {
                     for (deviceId in deviceIds) {
                         val olmSessionResult = results.getObject(userId, deviceId)
-                        if (olmSessionResult!!.sessionId != null) {
+                        if (olmSessionResult!!.sessionId != null && !force) {
                             // We already have a result for this device
                             continue
                         }
@@ -89,7 +90,7 @@ internal class EnsureOlmSessionsForDevicesAction @Inject constructor(private val
                             oneTimeKey = key
                         }
                         if (oneTimeKey == null) {
-                            Timber.v("## ensureOlmSessionsForDevices() : No one-time keys " + oneTimeKeyAlgorithm
+                            Timber.v("## CRYPTO | ensureOlmSessionsForDevices() : No one-time keys " + oneTimeKeyAlgorithm
                                     + " for device " + userId + " : " + deviceId)
                             continue
                         }
@@ -125,14 +126,14 @@ internal class EnsureOlmSessionsForDevicesAction @Inject constructor(private val
                 sessionId = olmDevice.createOutboundSession(deviceInfo.identityKey()!!, oneTimeKey.value)
 
                 if (!sessionId.isNullOrEmpty()) {
-                    Timber.v("## verifyKeyAndStartSession() : Started new sessionid " + sessionId
+                    Timber.v("## CRYPTO | verifyKeyAndStartSession() : Started new sessionid " + sessionId
                             + " for device " + deviceInfo + "(theirOneTimeKey: " + oneTimeKey.value + ")")
                 } else {
                     // Possibly a bad key
-                    Timber.e("## verifyKeyAndStartSession() : Error starting session with device $userId:$deviceId")
+                    Timber.e("## CRYPTO | verifyKeyAndStartSession() : Error starting session with device $userId:$deviceId")
                 }
             } else {
-                Timber.e("## verifyKeyAndStartSession() : Unable to verify signature on one-time key for device " + userId
+                Timber.e("## CRYPTO | verifyKeyAndStartSession() : Unable to verify signature on one-time key for device " + userId
                         + ":" + deviceId + " Error " + errorMessage)
             }
         }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/actions/MessageEncrypter.kt

@@ -16,19 +16,24 @@
 
 package im.vector.matrix.android.internal.crypto.actions
 
-import im.vector.matrix.android.api.auth.data.Credentials
+import im.vector.matrix.android.api.session.events.model.Content
 import im.vector.matrix.android.internal.crypto.MXCRYPTO_ALGORITHM_OLM
 import im.vector.matrix.android.internal.crypto.MXOlmDevice
 import im.vector.matrix.android.internal.crypto.model.CryptoDeviceInfo
 import im.vector.matrix.android.internal.crypto.model.rest.EncryptedMessage
+import im.vector.matrix.android.internal.di.DeviceId
+import im.vector.matrix.android.internal.di.UserId
 import im.vector.matrix.android.internal.util.JsonCanonicalizer
 import im.vector.matrix.android.internal.util.convertToUTF8
 import timber.log.Timber
 import javax.inject.Inject
 
-internal class MessageEncrypter @Inject constructor(private val credentials: Credentials,
-                                                    private val olmDevice: MXOlmDevice) {
-
+internal class MessageEncrypter @Inject constructor(
+        @UserId
+        private val userId: String,
+        @DeviceId
+        private val deviceId: String?,
+        private val olmDevice: MXOlmDevice) {
     /**
      * Encrypt an event payload for a list of devices.
      * This method must be called from the getCryptoHandler() thread.
@@ -37,13 +42,13 @@ internal class MessageEncrypter @Inject constructor(private val credentials: Cre
      * @param deviceInfos   list of device infos to encrypt for.
      * @return the content for an m.room.encrypted event.
      */
-    fun encryptMessage(payloadFields: Map<String, Any>, deviceInfos: List<CryptoDeviceInfo>): EncryptedMessage {
+    fun encryptMessage(payloadFields: Content, deviceInfos: List<CryptoDeviceInfo>): EncryptedMessage {
         val deviceInfoParticipantKey = deviceInfos.associateBy { it.identityKey()!! }
 
         val payloadJson = payloadFields.toMutableMap()
 
-        payloadJson["sender"] = credentials.userId
-        payloadJson["sender_device"] = credentials.deviceId!!
+        payloadJson["sender"] = userId
+        payloadJson["sender_device"] = deviceId!!
 
         // Include the Ed25519 key so that the recipient knows what
         // device this message came from.
@@ -53,11 +58,9 @@ internal class MessageEncrypter @Inject constructor(private val credentials: Cre
         // homeserver signed by the ed25519 key this proves that
         // the curve25519 key and the ed25519 key are owned by
         // the same device.
-        val keysMap = HashMap<String, String>()
-        keysMap["ed25519"] = olmDevice.deviceEd25519Key!!
-        payloadJson["keys"] = keysMap
+        payloadJson["keys"] = mapOf("ed25519" to olmDevice.deviceEd25519Key!!)
 
-        val ciphertext = HashMap<String, Any>()
+        val ciphertext = mutableMapOf<String, Any>()
 
         for ((deviceKey, deviceInfo) in deviceInfoParticipantKey) {
             val sessionId = olmDevice.getSessionId(deviceKey)

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/actions/SetDeviceVerificationAction.kt

@@ -48,7 +48,7 @@ internal class SetDeviceVerificationAction @Inject constructor(
 
         if (device.trustLevel != trustLevel) {
             device.trustLevel = trustLevel
-            cryptoStore.storeUserDevice(userId, device)
+            cryptoStore.setDeviceTrust(userId, deviceId, trustLevel.crossSigningVerified, trustLevel.locallyVerified)
         }
     }
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/algorithms/IMXDecrypting.kt

@@ -17,6 +17,7 @@
 
 package im.vector.matrix.android.internal.crypto.algorithms
 
+import im.vector.matrix.android.api.session.crypto.MXCryptoError
 import im.vector.matrix.android.api.session.events.model.Event
 import im.vector.matrix.android.internal.crypto.IncomingRoomKeyRequest
 import im.vector.matrix.android.internal.crypto.IncomingSecretShareRequest
@@ -35,6 +36,7 @@ internal interface IMXDecrypting {
      * @param timeline the id of the timeline where the event is decrypted. It is used to prevent replay attack.
      * @return the decryption information, or an error
      */
+    @Throws(MXCryptoError::class)
     fun decryptEvent(event: Event, timeline: String): MXEventDecryptionResult
 
     /**

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/algorithms/IMXEncrypting.kt

@@ -33,4 +33,34 @@ internal interface IMXEncrypting {
      * @return the encrypted content
      */
     suspend fun encryptEventContent(eventContent: Content, eventType: String, userIds: List<String>): Content
+
+    /**
+     * In Megolm, each recipient maintains a record of the ratchet value which allows
+     * them to decrypt any messages sent in the session after the corresponding point
+     * in the conversation. If this value is compromised, an attacker can similarly
+     * decrypt past messages which were encrypted by a key derived from the
+     * compromised or subsequent ratchet values. This gives 'partial' forward
+     * secrecy.
+     *
+     * To mitigate this issue, the application should offer the user the option to
+     * discard historical conversations, by winding forward any stored ratchet values,
+     * or discarding sessions altogether.
+     */
+    fun discardSessionKey()
+
+    /**
+     * Re-shares a session key with devices if the key has already been
+     * sent to them.
+     *
+     * @param sessionId The id of the outbound session to share.
+     * @param userId The id of the user who owns the target device.
+     * @param deviceId The id of the target device.
+     * @param senderKey The key of the originating device for the session.
+     *
+     * @return true in case of success
+     */
+    suspend fun reshareKey(sessionId: String,
+                           userId: String,
+                           deviceId: String,
+                           senderKey: String): Boolean
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/algorithms/megolm/MXMegolmDecryption.kt

@@ -63,6 +63,7 @@ internal class MXMegolmDecryption(private val userId: String,
      */
     private var pendingEvents: MutableMap<String /* senderKey|sessionId */, MutableMap<String /* timelineId */, MutableList<Event>>> = HashMap()
 
+    @Throws(MXCryptoError::class)
     override fun decryptEvent(event: Event, timeline: String): MXEventDecryptionResult {
         // If cross signing is enabled, we don't send request until the keys are trusted
         // There could be a race effect here when xsigning is enabled, we should ensure that keys was downloaded once
@@ -70,7 +71,9 @@ internal class MXMegolmDecryption(private val userId: String,
         return decryptEvent(event, timeline, requestOnFail)
     }
 
+    @Throws(MXCryptoError::class)
     private fun decryptEvent(event: Event, timeline: String, requestKeysOnFail: Boolean): MXEventDecryptionResult {
+        Timber.v("## CRYPTO | decryptEvent ${event.eventId} , requestKeysOnFail:$requestKeysOnFail")
         if (event.roomId.isNullOrBlank()) {
             throw MXCryptoError.Base(MXCryptoError.ErrorType.MISSING_FIELDS, MXCryptoError.MISSING_FIELDS_REASON)
         }
@@ -188,7 +191,7 @@ internal class MXMegolmDecryption(private val userId: String,
         val events = timeline.getOrPut(timelineId) { ArrayList() }
 
         if (event !in events) {
-            Timber.v("## addEventToPendingList() : add Event ${event.eventId} in room id ${event.roomId}")
+            Timber.v("## CRYPTO | addEventToPendingList() : add Event ${event.eventId} in room id ${event.roomId}")
             events.add(event)
         }
     }
@@ -199,6 +202,7 @@ internal class MXMegolmDecryption(private val userId: String,
      * @param event the key event.
      */
     override fun onRoomKeyEvent(event: Event, defaultKeysBackupService: DefaultKeysBackupService) {
+        Timber.v("## CRYPTO | onRoomKeyEvent()")
         var exportFormat = false
         val roomKeyContent = event.getClearContent().toModel<RoomKeyContent>() ?: return
 
@@ -207,11 +211,11 @@ internal class MXMegolmDecryption(private val userId: String,
         val forwardingCurve25519KeyChain: MutableList<String> = ArrayList()
 
         if (roomKeyContent.roomId.isNullOrEmpty() || roomKeyContent.sessionId.isNullOrEmpty() || roomKeyContent.sessionKey.isNullOrEmpty()) {
-            Timber.e("## onRoomKeyEvent() :  Key event is missing fields")
+            Timber.e("## CRYPTO | onRoomKeyEvent() :  Key event is missing fields")
             return
         }
         if (event.getClearType() == EventType.FORWARDED_ROOM_KEY) {
-            Timber.v("## onRoomKeyEvent(), forward adding key : roomId ${roomKeyContent.roomId}" +
+            Timber.v("## CRYPTO | onRoomKeyEvent(), forward adding key : roomId ${roomKeyContent.roomId}" +
                     " sessionId ${roomKeyContent.sessionId} sessionKey ${roomKeyContent.sessionKey}")
             val forwardedRoomKeyContent = event.getClearContent().toModel<ForwardedRoomKeyContent>()
                     ?: return
@@ -221,7 +225,7 @@ internal class MXMegolmDecryption(private val userId: String,
             }
 
             if (senderKey == null) {
-                Timber.e("## onRoomKeyEvent() : event is missing sender_key field")
+                Timber.e("## CRYPTO | onRoomKeyEvent() : event is missing sender_key field")
                 return
             }
 
@@ -230,18 +234,18 @@ internal class MXMegolmDecryption(private val userId: String,
             exportFormat = true
             senderKey = forwardedRoomKeyContent.senderKey
             if (null == senderKey) {
-                Timber.e("## onRoomKeyEvent() : forwarded_room_key event is missing sender_key field")
+                Timber.e("## CRYPTO | onRoomKeyEvent() : forwarded_room_key event is missing sender_key field")
                 return
             }
 
             if (null == forwardedRoomKeyContent.senderClaimedEd25519Key) {
-                Timber.e("## forwarded_room_key_event is missing sender_claimed_ed25519_key field")
+                Timber.e("## CRYPTO | forwarded_room_key_event is missing sender_claimed_ed25519_key field")
                 return
             }
 
             keysClaimed["ed25519"] = forwardedRoomKeyContent.senderClaimedEd25519Key
         } else {
-            Timber.v("## onRoomKeyEvent(), Adding key : roomId " + roomKeyContent.roomId + " sessionId " + roomKeyContent.sessionId
+            Timber.v("## CRYPTO | onRoomKeyEvent(), Adding key : roomId " + roomKeyContent.roomId + " sessionId " + roomKeyContent.sessionId
                     + " sessionKey " + roomKeyContent.sessionKey) // from " + event);
 
             if (null == senderKey) {
@@ -253,6 +257,7 @@ internal class MXMegolmDecryption(private val userId: String,
             keysClaimed = event.getKeysClaimed().toMutableMap()
         }
 
+        Timber.e("## CRYPTO | onRoomKeyEvent addInboundGroupSession ${roomKeyContent.sessionId}")
         val added = olmDevice.addInboundGroupSession(roomKeyContent.sessionId,
                 roomKeyContent.sessionKey,
                 roomKeyContent.roomId,
@@ -284,7 +289,7 @@ internal class MXMegolmDecryption(private val userId: String,
      * @param sessionId the session id
      */
     override fun onNewSession(senderKey: String, sessionId: String) {
-        Timber.v("ON NEW SESSION $sessionId - $senderKey")
+        Timber.v(" CRYPTO | ON NEW SESSION $sessionId - $senderKey")
         newSessionListener?.onNewSession(null, senderKey, sessionId)
     }
 
@@ -318,7 +323,7 @@ internal class MXMegolmDecryption(private val userId: String,
                                 // were no one-time keys.
                                 return@mapCatching
                             }
-                            Timber.v("## shareKeysWithDevice() : sharing keys for session" +
+                            Timber.v("## CRYPTO | shareKeysWithDevice() : sharing keys for session" +
                                     " ${body.senderKey}|${body.sessionId} with device $userId:$deviceId")
 
                             val payloadJson = mutableMapOf<String, Any>("type" to EventType.FORWARDED_ROOM_KEY)
@@ -337,7 +342,7 @@ internal class MXMegolmDecryption(private val userId: String,
                             val encodedPayload = messageEncrypter.encryptMessage(payloadJson, listOf(deviceInfo))
                             val sendToDeviceMap = MXUsersDevicesMap<Any>()
                             sendToDeviceMap.setObject(userId, deviceId, encodedPayload)
-                            Timber.v("## shareKeysWithDevice() : sending to $userId:$deviceId")
+                            Timber.v("## CRYPTO | shareKeysWithDevice() : sending to $userId:$deviceId")
                             val sendToDeviceParams = SendToDeviceTask.Params(EventType.ENCRYPTED, sendToDeviceMap)
                             sendToDeviceTask.execute(sendToDeviceParams)
                         }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/algorithms/megolm/MXMegolmEncryption.kt

@@ -40,7 +40,7 @@ import timber.log.Timber
 
 internal class MXMegolmEncryption(
         // The id of the room we will be sending to.
-        private var roomId: String,
+        private val roomId: String,
         private val olmDevice: MXOlmDevice,
         private val defaultKeysBackupService: DefaultKeysBackupService,
         private val cryptoStore: IMXCryptoStore,
@@ -66,17 +66,25 @@ internal class MXMegolmEncryption(
     override suspend fun encryptEventContent(eventContent: Content,
                                              eventType: String,
                                              userIds: List<String>): Content {
+        val ts = System.currentTimeMillis()
+        Timber.v("## CRYPTO | encryptEventContent : getDevicesInRoom")
         val devices = getDevicesInRoom(userIds)
+        Timber.v("## CRYPTO | encryptEventContent ${System.currentTimeMillis() - ts}: getDevicesInRoom ${devices.map}")
         val outboundSession = ensureOutboundSession(devices)
         return encryptContent(outboundSession, eventType, eventContent)
     }
 
+    override fun discardSessionKey() {
+        outboundSession = null
+    }
+
     /**
      * Prepare a new session.
      *
      * @return the session description
      */
     private fun prepareNewSessionInRoom(): MXOutboundSessionInfo {
+        Timber.v("## CRYPTO | prepareNewSessionInRoom() ")
         val sessionId = olmDevice.createOutboundGroupSession()
 
         val keysClaimedMap = HashMap<String, String>()
@@ -96,6 +104,7 @@ internal class MXMegolmEncryption(
      * @param devicesInRoom the devices list
      */
     private suspend fun ensureOutboundSession(devicesInRoom: MXUsersDevicesMap<CryptoDeviceInfo>): MXOutboundSessionInfo {
+        Timber.v("## CRYPTO | ensureOutboundSession start")
         var session = outboundSession
         if (session == null
                 // Need to make a brand new session?
@@ -132,7 +141,7 @@ internal class MXMegolmEncryption(
                                  devicesByUsers: Map<String, List<CryptoDeviceInfo>>) {
         // nothing to send, the task is done
         if (devicesByUsers.isEmpty()) {
-            Timber.v("## shareKey() : nothing more to do")
+            Timber.v("## CRYPTO | shareKey() : nothing more to do")
             return
         }
         // reduce the map size to avoid request timeout when there are too many devices (Users size  * devices per user)
@@ -145,7 +154,7 @@ internal class MXMegolmEncryption(
                 break
             }
         }
-        Timber.v("## shareKey() ; userId ${subMap.keys}")
+        Timber.v("## CRYPTO | shareKey() ; sessionId<${session.sessionId}> userId ${subMap.keys}")
         shareUserDevicesKey(session, subMap)
         val remainingDevices = devicesByUsers - subMap.keys
         shareKey(session, remainingDevices)
@@ -174,10 +183,10 @@ internal class MXMegolmEncryption(
         payload["content"] = submap
 
         var t0 = System.currentTimeMillis()
-        Timber.v("## shareUserDevicesKey() : starts")
+        Timber.v("## CRYPTO | shareUserDevicesKey() : starts")
 
         val results = ensureOlmSessionsForDevicesAction.handle(devicesByUser)
-        Timber.v("## shareUserDevicesKey() : ensureOlmSessionsForDevices succeeds after "
+        Timber.v("## CRYPTO | shareUserDevicesKey() : ensureOlmSessionsForDevices succeeds after "
                 + (System.currentTimeMillis() - t0) + " ms")
         val contentMap = MXUsersDevicesMap<Any>()
         var haveTargets = false
@@ -200,17 +209,17 @@ internal class MXMegolmEncryption(
                     // so just skip it.
                     continue
                 }
-                Timber.v("## shareUserDevicesKey() : Sharing keys with device $userId:$deviceID")
+                Timber.v("## CRYPTO | shareUserDevicesKey() : Sharing keys with device $userId:$deviceID")
                 contentMap.setObject(userId, deviceID, messageEncrypter.encryptMessage(payload, listOf(sessionResult.deviceInfo)))
                 haveTargets = true
             }
         }
         if (haveTargets) {
             t0 = System.currentTimeMillis()
-            Timber.v("## shareUserDevicesKey() : has target")
+            Timber.v("## CRYPTO | shareUserDevicesKey() : has target")
             val sendToDeviceParams = SendToDeviceTask.Params(EventType.ENCRYPTED, contentMap)
             sendToDeviceTask.execute(sendToDeviceParams)
-            Timber.v("## shareUserDevicesKey() : sendToDevice succeeds after "
+            Timber.v("## CRYPTO | shareUserDevicesKey() : sendToDevice succeeds after "
                     + (System.currentTimeMillis() - t0) + " ms")
 
             // Add the devices we have shared with to session.sharedWithDevices.
@@ -224,7 +233,7 @@ internal class MXMegolmEncryption(
                 }
             }
         } else {
-            Timber.v("## shareUserDevicesKey() : no need to sharekey")
+            Timber.v("## CRYPTO | shareUserDevicesKey() : no need to sharekey")
         }
     }
 
@@ -305,4 +314,49 @@ internal class MXMegolmEncryption(
             throw MXCryptoError.UnknownDevice(unknownDevices)
         }
     }
+
+    override suspend fun reshareKey(sessionId: String,
+                                    userId: String,
+                                    deviceId: String,
+                                    senderKey: String): Boolean {
+        Timber.d("[MXMegolmEncryption] reshareKey: $sessionId to $userId:$deviceId")
+        val deviceInfo = cryptoStore.getUserDevice(userId, deviceId) ?: return false
+                .also { Timber.w("Device not found") }
+
+        // Get the chain index of the key we previously sent this device
+        val chainIndex = outboundSession?.sharedWithDevices?.getObject(userId, deviceId)?.toLong() ?: return false
+                .also { Timber.w("[MXMegolmEncryption] reshareKey : ERROR : Never share megolm with this device") }
+
+        val devicesByUser = mapOf(userId to listOf(deviceInfo))
+        val usersDeviceMap = ensureOlmSessionsForDevicesAction.handle(devicesByUser)
+        val olmSessionResult = usersDeviceMap.getObject(userId, deviceId)
+        olmSessionResult?.sessionId
+                ?: // no session with this device, probably because there were no one-time keys.
+                // ensureOlmSessionsForDevicesAction has already done the logging, so just skip it.
+                return false
+
+        Timber.d("[MXMegolmEncryption] reshareKey: sharing keys for session $senderKey|$sessionId:$chainIndex with device $userId:$deviceId")
+
+        val payloadJson = mutableMapOf<String, Any>("type" to EventType.FORWARDED_ROOM_KEY)
+
+        runCatching { olmDevice.getInboundGroupSession(sessionId, senderKey, roomId) }
+                .fold(
+                        {
+                            // TODO
+                            payloadJson["content"] = it.exportKeys(chainIndex) ?: ""
+                        },
+                        {
+                            // TODO
+                        }
+
+                )
+
+        val encodedPayload = messageEncrypter.encryptMessage(payloadJson, listOf(deviceInfo))
+        val sendToDeviceMap = MXUsersDevicesMap<Any>()
+        sendToDeviceMap.setObject(userId, deviceId, encodedPayload)
+        Timber.v("## CRYPTO | CRYPTO | shareKeysWithDevice() : sending to $userId:$deviceId")
+        val sendToDeviceParams = SendToDeviceTask.Params(EventType.ENCRYPTED, sendToDeviceMap)
+        sendToDeviceTask.execute(sendToDeviceParams)
+        return true
+    }
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/algorithms/olm/MXOlmDecryption.kt

@@ -38,6 +38,7 @@ internal class MXOlmDecryption(
         private val userId: String)
     : IMXDecrypting {
 
+    @Throws(MXCryptoError::class)
     override fun decryptEvent(event: Event, timeline: String): MXEventDecryptionResult {
         val olmEventContent = event.content.toModel<OlmEventContent>() ?: run {
             Timber.e("## decryptEvent() : bad event format")

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/algorithms/olm/MXOlmEncryption.kt

@@ -29,7 +29,7 @@ import im.vector.matrix.android.internal.crypto.model.CryptoDeviceInfo
 import im.vector.matrix.android.internal.crypto.store.IMXCryptoStore
 
 internal class MXOlmEncryption(
-        private var roomId: String,
+        private val roomId: String,
         private val olmDevice: MXOlmDevice,
         private val cryptoStore: IMXCryptoStore,
         private val messageEncrypter: MessageEncrypter,
@@ -78,4 +78,13 @@ internal class MXOlmEncryption(
         deviceListManager.downloadKeys(users, false)
         ensureOlmSessionsForUsersAction.handle(users)
     }
+
+    override fun discardSessionKey() {
+        // No need for olm
+    }
+
+    override suspend fun reshareKey(sessionId: String, userId: String, deviceId: String, senderKey: String): Boolean {
+        // No need for olm
+        return false
+    }
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/crosssigning/ComputeTrustTask.kt

@@ -19,6 +19,7 @@ import im.vector.matrix.android.api.crypto.RoomEncryptionTrustLevel
 import im.vector.matrix.android.api.extensions.orFalse
 import im.vector.matrix.android.api.session.crypto.crosssigning.MXCrossSigningInfo
 import im.vector.matrix.android.internal.crypto.store.IMXCryptoStore
+import im.vector.matrix.android.internal.di.UserId
 import im.vector.matrix.android.internal.task.Task
 import im.vector.matrix.android.internal.util.MatrixCoroutineDispatchers
 import kotlinx.coroutines.withContext
@@ -26,17 +27,28 @@ import javax.inject.Inject
 
 internal interface ComputeTrustTask : Task<ComputeTrustTask.Params, RoomEncryptionTrustLevel> {
     data class Params(
-            val userIds: List<String>
+            val activeMemberUserIds: List<String>,
+            val isDirectRoom: Boolean
     )
 }
 
 internal class DefaultComputeTrustTask @Inject constructor(
         private val cryptoStore: IMXCryptoStore,
+        @UserId private val userId: String,
         private val coroutineDispatchers: MatrixCoroutineDispatchers
 ) : ComputeTrustTask {
 
     override suspend fun execute(params: ComputeTrustTask.Params): RoomEncryptionTrustLevel = withContext(coroutineDispatchers.crypto) {
-        val allTrustedUserIds = params.userIds
+        // The set of “all users” depends on the type of room:
+        // For regular / topic rooms, all users including yourself, are considered when decorating a room
+        // For 1:1 and group DM rooms, all other users (i.e. excluding yourself) are considered when decorating a room
+        val listToCheck = if (params.isDirectRoom) {
+            params.activeMemberUserIds.filter { it != userId }
+        } else {
+            params.activeMemberUserIds
+        }
+
+        val allTrustedUserIds = listToCheck
                 .filter { userId -> getUserCrossSigningKeys(userId)?.isTrusted() == true }
 
         if (allTrustedUserIds.isEmpty()) {
@@ -60,7 +72,7 @@ internal class DefaultComputeTrustTask @Inject constructor(
                         if (hasWarning) {
                             RoomEncryptionTrustLevel.Warning
                         } else {
-                            if (params.userIds.size == allTrustedUserIds.size) {
+                            if (listToCheck.size == allTrustedUserIds.size) {
                                 // all users are trusted and all devices are verified
                                 RoomEncryptionTrustLevel.Trusted
                             } else {

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/crosssigning/DefaultCrossSigningService.kt

@@ -470,6 +470,10 @@ internal class DefaultCrossSigningService @Inject constructor(
         return cryptoStore.getCrossSigningPrivateKeys()
     }
 
+    override fun getLiveCrossSigningPrivateKeys(): LiveData<Optional<PrivateKeysInfo>> {
+        return cryptoStore.getLiveCrossSigningPrivateKeys()
+    }
+
     override fun canCrossSign(): Boolean {
         return checkSelfTrust().isVerified() && cryptoStore.getCrossSigningPrivateKeys()?.selfSigned != null
                 && cryptoStore.getCrossSigningPrivateKeys()?.user != null

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/crosssigning/SessionToCryptoRoomMembersUpdate.kt

@@ -17,6 +17,7 @@ package im.vector.matrix.android.internal.crypto.crosssigning
 
 data class SessionToCryptoRoomMembersUpdate(
         val roomId: String,
+        val isDirect: Boolean,
         val userIds: List<String>
 )
 

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/crosssigning/ShieldTrustUpdater.kt

@@ -15,18 +15,20 @@
  */
 package im.vector.matrix.android.internal.crypto.crosssigning
 
+import im.vector.matrix.android.api.extensions.orFalse
 import im.vector.matrix.android.internal.database.model.RoomMemberSummaryEntity
 import im.vector.matrix.android.internal.database.model.RoomMemberSummaryEntityFields
+import im.vector.matrix.android.internal.database.model.RoomSummaryEntity
 import im.vector.matrix.android.internal.database.query.where
 import im.vector.matrix.android.internal.di.SessionDatabase
 import im.vector.matrix.android.internal.session.room.RoomSummaryUpdater
+import im.vector.matrix.android.internal.session.room.membership.RoomMemberHelper
 import im.vector.matrix.android.internal.task.TaskExecutor
-import im.vector.matrix.android.internal.util.MatrixCoroutineDispatchers
 import im.vector.matrix.android.internal.util.createBackgroundHandler
 import io.realm.Realm
 import io.realm.RealmConfiguration
+import kotlinx.coroutines.android.asCoroutineDispatcher
 import kotlinx.coroutines.launch
-import kotlinx.coroutines.withContext
 import org.greenrobot.eventbus.EventBus
 import org.greenrobot.eventbus.Subscribe
 import timber.log.Timber
@@ -38,13 +40,13 @@ internal class ShieldTrustUpdater @Inject constructor(
         private val eventBus: EventBus,
         private val computeTrustTask: ComputeTrustTask,
         private val taskExecutor: TaskExecutor,
-        private val coroutineDispatchers: MatrixCoroutineDispatchers,
         @SessionDatabase private val sessionRealmConfiguration: RealmConfiguration,
         private val roomSummaryUpdater: RoomSummaryUpdater
 ) {
 
     companion object {
         private val BACKGROUND_HANDLER = createBackgroundHandler("SHIELD_CRYPTO_DB_THREAD")
+        private val BACKGROUND_HANDLER_DISPATCHER = BACKGROUND_HANDLER.asCoroutineDispatcher()
     }
 
     private val backgroundSessionRealm = AtomicReference<Realm>()
@@ -76,14 +78,11 @@ internal class ShieldTrustUpdater @Inject constructor(
         if (!isStarted.get()) {
             return
         }
-        taskExecutor.executorScope.launch(coroutineDispatchers.crypto) {
-            val updatedTrust = computeTrustTask.execute(ComputeTrustTask.Params(update.userIds))
+        taskExecutor.executorScope.launch(BACKGROUND_HANDLER_DISPATCHER) {
+            val updatedTrust = computeTrustTask.execute(ComputeTrustTask.Params(update.userIds, update.isDirect))
             // We need to send that back to session base
-
-            BACKGROUND_HANDLER.post {
-                backgroundSessionRealm.get()?.executeTransaction { realm ->
-                    roomSummaryUpdater.updateShieldTrust(realm, update.roomId, updatedTrust)
-                }
+            backgroundSessionRealm.get()?.executeTransaction { realm ->
+                roomSummaryUpdater.updateShieldTrust(realm, update.roomId, updatedTrust)
             }
         }
     }
@@ -93,45 +92,31 @@ internal class ShieldTrustUpdater @Inject constructor(
         if (!isStarted.get()) {
             return
         }
-
         onCryptoDevicesChange(update.userIds)
     }
 
     private fun onCryptoDevicesChange(users: List<String>) {
-        BACKGROUND_HANDLER.post {
-            val impactedRoomsId = backgroundSessionRealm.get()?.where(RoomMemberSummaryEntity::class.java)
-                    ?.`in`(RoomMemberSummaryEntityFields.USER_ID, users.toTypedArray())
-                    ?.findAll()
-                    ?.map { it.roomId }
-                    ?.distinct()
+        taskExecutor.executorScope.launch(BACKGROUND_HANDLER_DISPATCHER) {
+            val realm = backgroundSessionRealm.get() ?: return@launch
+            val distinctRoomIds = realm.where(RoomMemberSummaryEntity::class.java)
+                    .`in`(RoomMemberSummaryEntityFields.USER_ID, users.toTypedArray())
+                    .distinct(RoomMemberSummaryEntityFields.ROOM_ID)
+                    .findAll()
+                    .map { it.roomId }
 
-            val map = HashMap<String, List<String>>()
-            impactedRoomsId?.forEach { roomId ->
-                backgroundSessionRealm.get()?.let { realm ->
-                    RoomMemberSummaryEntity.where(realm, roomId)
-                            .findAll()
-                            .let { results ->
-                                map[roomId] = results.map { it.userId }
-                            }
-                }
-            }
-
-            map.forEach { entry ->
-                val roomId = entry.key
-                val userList = entry.value
-                taskExecutor.executorScope.launch {
-                    withContext(coroutineDispatchers.crypto) {
-                        try {
-                            // Can throw if the crypto database has been closed in between, in this case log and ignore?
-                            val updatedTrust = computeTrustTask.execute(ComputeTrustTask.Params(userList))
-                            BACKGROUND_HANDLER.post {
-                                backgroundSessionRealm.get()?.executeTransaction { realm ->
-                                    roomSummaryUpdater.updateShieldTrust(realm, roomId, updatedTrust)
-                                }
-                            }
-                        } catch (failure: Throwable) {
-                            Timber.e(failure)
+            distinctRoomIds.forEach { roomId ->
+                val roomSummary = RoomSummaryEntity.where(realm, roomId).findFirst()
+                if (roomSummary?.isEncrypted.orFalse()) {
+                    val allActiveRoomMembers = RoomMemberHelper(realm, roomId).getActiveRoomMemberIds()
+                    try {
+                        val updatedTrust = computeTrustTask.execute(
+                                ComputeTrustTask.Params(allActiveRoomMembers, roomSummary?.isDirect == true)
+                        )
+                        realm.executeTransaction {
+                            roomSummaryUpdater.updateShieldTrust(it, roomId, updatedTrust)
                         }
+                    } catch (failure: Throwable) {
+                        Timber.e(failure)
                     }
                 }
             }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/keysbackup/DefaultKeysBackupService.kt

@@ -66,7 +66,7 @@ import im.vector.matrix.android.internal.crypto.keysbackup.tasks.UpdateKeysBacku
 import im.vector.matrix.android.internal.crypto.keysbackup.util.computeRecoveryKey
 import im.vector.matrix.android.internal.crypto.keysbackup.util.extractCurveKeyFromRecoveryKey
 import im.vector.matrix.android.internal.crypto.model.ImportRoomKeysResult
-import im.vector.matrix.android.internal.crypto.model.OlmInboundGroupSessionWrapper
+import im.vector.matrix.android.internal.crypto.model.OlmInboundGroupSessionWrapper2
 import im.vector.matrix.android.internal.crypto.store.IMXCryptoStore
 import im.vector.matrix.android.internal.crypto.store.SavedKeyBackupKeyInfo
 import im.vector.matrix.android.internal.crypto.store.db.model.KeysBackupDataEntity
@@ -1318,7 +1318,7 @@ internal class DefaultKeysBackupService @Inject constructor(
 
     @VisibleForTesting
     @WorkerThread
-    fun encryptGroupSession(olmInboundGroupSessionWrapper: OlmInboundGroupSessionWrapper): KeyBackupData {
+    fun encryptGroupSession(olmInboundGroupSessionWrapper: OlmInboundGroupSessionWrapper2): KeyBackupData {
         // Gather information for each key
         val device = cryptoStore.deviceWithIdentityKey(olmInboundGroupSessionWrapper.senderKey!!)
 

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/model/CryptoDeviceInfo.kt

@@ -29,7 +29,8 @@ data class CryptoDeviceInfo(
         override val signatures: Map<String, Map<String, String>>? = null,
         val unsigned: JsonDict? = null,
         var trustLevel: DeviceTrustLevel? = null,
-        var isBlocked: Boolean = false
+        var isBlocked: Boolean = false,
+        val firstTimeSeenLocalTs: Long? = null
 ) : CryptoInfo {
 
     val isVerified: Boolean

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/model/CryptoInfoMapper.kt

@@ -61,20 +61,4 @@ internal object CryptoInfoMapper {
                 signatures = keyInfo.signatures
         )
     }
-
-    fun RestDeviceInfo.toCryptoModel(): CryptoDeviceInfo {
-        return map(this)
-    }
-
-    fun CryptoDeviceInfo.toRest(): RestDeviceInfo {
-        return map(this)
-    }
-
-//    fun RestKeyInfo.toCryptoModel(): CryptoCrossSigningKey {
-//        return map(this)
-//    }
-
-    fun CryptoCrossSigningKey.toRest(): RestKeyInfo {
-        return map(this)
-    }
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/model/OlmInboundGroupSessionWrapper2.kt

@@ -0,0 +1,157 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package im.vector.matrix.android.internal.crypto.model
+
+import im.vector.matrix.android.internal.crypto.MXCRYPTO_ALGORITHM_MEGOLM
+import im.vector.matrix.android.internal.crypto.MegolmSessionData
+import org.matrix.olm.OlmInboundGroupSession
+import timber.log.Timber
+import java.io.Serializable
+
+/**
+ * This class adds more context to a OlmInboundGroupSession object.
+ * This allows additional checks. The class implements Serializable so that the context can be stored.
+ */
+class OlmInboundGroupSessionWrapper2 : Serializable {
+
+    // The associated olm inbound group session.
+    var olmInboundGroupSession: OlmInboundGroupSession? = null
+
+    // The room in which this session is used.
+    var roomId: String? = null
+
+    // The base64-encoded curve25519 key of the sender.
+    var senderKey: String? = null
+
+    // Other keys the sender claims.
+    var keysClaimed: Map<String, String>? = null
+
+    // Devices which forwarded this session to us (normally empty).
+    var forwardingCurve25519KeyChain: List<String>? = ArrayList()
+
+    /**
+     * @return the first known message index
+     */
+    val firstKnownIndex: Long?
+        get() {
+            if (null != olmInboundGroupSession) {
+                try {
+                    return olmInboundGroupSession!!.firstKnownIndex
+                } catch (e: Exception) {
+                    Timber.e(e, "## getFirstKnownIndex() : getFirstKnownIndex failed")
+                }
+            }
+
+            return null
+        }
+
+    /**
+     * Constructor
+     *
+     * @param sessionKey the session key
+     * @param isImported true if it is an imported session key
+     */
+    constructor(sessionKey: String, isImported: Boolean) {
+        try {
+            if (!isImported) {
+                olmInboundGroupSession = OlmInboundGroupSession(sessionKey)
+            } else {
+                olmInboundGroupSession = OlmInboundGroupSession.importSession(sessionKey)
+            }
+        } catch (e: Exception) {
+            Timber.e(e, "Cannot create")
+        }
+    }
+
+    constructor() {
+        // empty
+    }
+    /**
+     * Create a new instance from the provided keys map.
+     *
+     * @param megolmSessionData the megolm session data
+     * @throws Exception if the data are invalid
+     */
+    @Throws(Exception::class)
+    constructor(megolmSessionData: MegolmSessionData) {
+        try {
+            olmInboundGroupSession = OlmInboundGroupSession.importSession(megolmSessionData.sessionKey!!)
+
+            if (olmInboundGroupSession!!.sessionIdentifier() != megolmSessionData.sessionId) {
+                throw Exception("Mismatched group session Id")
+            }
+
+            senderKey = megolmSessionData.senderKey
+            keysClaimed = megolmSessionData.senderClaimedKeys
+            roomId = megolmSessionData.roomId
+        } catch (e: Exception) {
+            throw Exception(e.message)
+        }
+    }
+
+    /**
+     * Export the inbound group session keys
+     * @param index the index to export. If null, the first known index will be used
+     *
+     * @return the inbound group session as MegolmSessionData if the operation succeeds
+     */
+    fun exportKeys(index: Long? = null): MegolmSessionData? {
+        return try {
+            if (null == forwardingCurve25519KeyChain) {
+                forwardingCurve25519KeyChain = ArrayList()
+            }
+
+            if (keysClaimed == null) {
+                return null
+            }
+
+            val wantedIndex = index ?: olmInboundGroupSession!!.firstKnownIndex
+
+            MegolmSessionData(
+                    senderClaimedEd25519Key = keysClaimed?.get("ed25519"),
+                    forwardingCurve25519KeyChain = ArrayList(forwardingCurve25519KeyChain!!),
+                    senderKey = senderKey,
+                    senderClaimedKeys = keysClaimed,
+                    roomId = roomId,
+                    sessionId = olmInboundGroupSession!!.sessionIdentifier(),
+                    sessionKey = olmInboundGroupSession!!.export(wantedIndex),
+                    algorithm = MXCRYPTO_ALGORITHM_MEGOLM
+            )
+        } catch (e: Exception) {
+            Timber.e(e, "## export() : senderKey $senderKey failed")
+            null
+        }
+    }
+
+    /**
+     * Export the session for a message index.
+     *
+     * @param messageIndex the message index
+     * @return the exported data
+     */
+    fun exportSession(messageIndex: Long): String? {
+        if (null != olmInboundGroupSession) {
+            try {
+                return olmInboundGroupSession!!.export(messageIndex)
+            } catch (e: Exception) {
+                Timber.e(e, "## exportSession() : export failed")
+            }
+        }
+
+        return null
+    }
+}

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/model/rest/DummyContent.kt

@@ -0,0 +1,22 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package im.vector.matrix.android.internal.crypto.model.rest
+
+/**
+ * Class representing the dummy content
+ * Ref: https://matrix.org/docs/spec/client_server/latest#id82
+ */
+typealias DummyContent = Unit

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/model/rest/ForwardedRoomKeyContent.kt

@@ -20,28 +20,53 @@ import com.squareup.moshi.JsonClass
 
 /**
  * Class representing the forward room key request body content
+ * Ref: https://matrix.org/docs/spec/client_server/latest#m-forwarded-room-key
  */
 @JsonClass(generateAdapter = true)
 data class ForwardedRoomKeyContent(
-
+        /**
+         * Required. The encryption algorithm the key in this event is to be used with.
+         */
         @Json(name = "algorithm")
         val algorithm: String? = null,
 
+        /**
+         * Required. The room where the key is used.
+         */
         @Json(name = "room_id")
         val roomId: String? = null,
 
+        /**
+         * Required. The Curve25519 key of the device which initiated the session originally.
+         */
         @Json(name = "sender_key")
         val senderKey: String? = null,
 
+        /**
+         * Required. The ID of the session that the key is for.
+         */
         @Json(name = "session_id")
         val sessionId: String? = null,
 
+        /**
+         * Required. The key to be exchanged.
+         */
         @Json(name = "session_key")
         val sessionKey: String? = null,
 
+        /**
+         * Required. Chain of Curve25519 keys. It starts out empty, but each time the key is forwarded to another device,
+         * the previous sender in the chain is added to the end of the list. For example, if the key is forwarded
+         * from A to B to C, this field is empty between A and B, and contains A's Curve25519 key between B and C.
+         */
         @Json(name = "forwarding_curve25519_key_chain")
         val forwardingCurve25519KeyChain: List<String>? = null,
 
+        /**
+         * Required. The Ed25519 key of the device which initiated the session originally. It is 'claimed' because the
+         * receiving device has no way to tell that the original room_key actually came from a device which owns the
+         * private part of this key unless they have done device verification.
+         */
         @Json(name = "sender_claimed_ed25519_key")
         val senderClaimedEd25519Key: String? = null
 )

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/secrets/DefaultSharedSecretStorageService.kt

@@ -419,7 +419,7 @@ internal class DefaultSharedSecretStorageService @Inject constructor(
                 ?: return IntegrityResult.Error(SharedSecretStorageError.UnknownKey(keyId ?: ""))
 
         if (keyInfo.content.algorithm != SSSS_ALGORITHM_AES_HMAC_SHA2
-                || keyInfo.content.algorithm != SSSS_ALGORITHM_CURVE25519_AES_SHA2) {
+                && keyInfo.content.algorithm != SSSS_ALGORITHM_CURVE25519_AES_SHA2) {
             // Unsupported algorithm
             return IntegrityResult.Error(
                     SharedSecretStorageError.UnsupportedAlgorithm(keyInfo.content.algorithm ?: "")

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/store/IMXCryptoStore.kt

@@ -30,8 +30,9 @@ import im.vector.matrix.android.internal.crypto.OutgoingRoomKeyRequest
 import im.vector.matrix.android.internal.crypto.OutgoingSecretRequest
 import im.vector.matrix.android.internal.crypto.model.CryptoCrossSigningKey
 import im.vector.matrix.android.internal.crypto.model.CryptoDeviceInfo
-import im.vector.matrix.android.internal.crypto.model.OlmInboundGroupSessionWrapper
+import im.vector.matrix.android.internal.crypto.model.OlmInboundGroupSessionWrapper2
 import im.vector.matrix.android.internal.crypto.model.OlmSessionWrapper
+import im.vector.matrix.android.internal.crypto.model.rest.DeviceInfo
 import im.vector.matrix.android.internal.crypto.model.rest.RoomKeyRequestBody
 import im.vector.matrix.android.internal.crypto.store.db.model.KeysBackupDataEntity
 import org.matrix.olm.OlmAccount
@@ -58,7 +59,7 @@ internal interface IMXCryptoStore {
      *
      * @return the list of all known group sessions, to export them.
      */
-    fun getInboundGroupSessions(): List<OlmInboundGroupSessionWrapper>
+    fun getInboundGroupSessions(): List<OlmInboundGroupSessionWrapper2>
 
     /**
      * @return true to unilaterally blacklist all unverified devices.
@@ -163,14 +164,6 @@ internal interface IMXCryptoStore {
      */
     fun saveOlmAccount()
 
-    /**
-     * Store a device for a user.
-     *
-     * @param userId the user's id.
-     * @param device the device to store.
-     */
-    fun storeUserDevice(userId: String?, deviceInfo: CryptoDeviceInfo?)
-
     /**
      * Retrieve a device for a user.
      *
@@ -196,7 +189,8 @@ internal interface IMXCryptoStore {
      */
     fun storeUserDevices(userId: String, devices: Map<String, CryptoDeviceInfo>?)
 
-    fun storeUserCrossSigningKeys(userId: String, masterKey: CryptoCrossSigningKey?,
+    fun storeUserCrossSigningKeys(userId: String,
+                                  masterKey: CryptoCrossSigningKey?,
                                   selfSigningKey: CryptoCrossSigningKey?,
                                   userSigningKey: CryptoCrossSigningKey?)
 
@@ -217,6 +211,9 @@ internal interface IMXCryptoStore {
     // TODO temp
     fun getLiveDeviceList(): LiveData<List<CryptoDeviceInfo>>
 
+    fun getMyDevicesInfo() : List<DeviceInfo>
+    fun getLiveMyDevicesInfo() : LiveData<List<DeviceInfo>>
+    fun saveMyDevicesInfo(info: List<DeviceInfo>)
     /**
      * Store the crypto algorithm for a room.
      *
@@ -262,7 +259,7 @@ internal interface IMXCryptoStore {
      * @param deviceKey the public key of the other device.
      * @return The Base64 end-to-end session, or null if not found
      */
-    fun getDeviceSession(sessionId: String?, deviceKey: String?): OlmSessionWrapper?
+    fun getDeviceSession(sessionId: String, deviceKey: String): OlmSessionWrapper?
 
     /**
      * Retrieve the last used sessionId, regarding `lastReceivedMessageTs`, or null if no session exist
@@ -277,7 +274,7 @@ internal interface IMXCryptoStore {
      *
      * @param sessions the inbound group sessions to store.
      */
-    fun storeInboundGroupSessions(sessions: List<OlmInboundGroupSessionWrapper>)
+    fun storeInboundGroupSessions(sessions: List<OlmInboundGroupSessionWrapper2>)
 
     /**
      * Retrieve an inbound group session.
@@ -286,7 +283,7 @@ internal interface IMXCryptoStore {
      * @param senderKey the base64-encoded curve25519 key of the sender.
      * @return an inbound group session.
      */
-    fun getInboundGroupSession(sessionId: String, senderKey: String): OlmInboundGroupSessionWrapper?
+    fun getInboundGroupSession(sessionId: String, senderKey: String): OlmInboundGroupSessionWrapper2?
 
     /**
      * Remove an inbound group session
@@ -310,7 +307,7 @@ internal interface IMXCryptoStore {
      *
      * @param sessions the sessions
      */
-    fun markBackupDoneForInboundGroupSessions(olmInboundGroupSessionWrappers: List<OlmInboundGroupSessionWrapper>)
+    fun markBackupDoneForInboundGroupSessions(olmInboundGroupSessionWrappers: List<OlmInboundGroupSessionWrapper2>)
 
     /**
      * Retrieve inbound group sessions that are not yet backed up.
@@ -318,7 +315,7 @@ internal interface IMXCryptoStore {
      * @param limit the maximum number of sessions to return.
      * @return an array of non backed up inbound group sessions.
      */
-    fun inboundGroupSessionsToBackup(limit: Int): List<OlmInboundGroupSessionWrapper>
+    fun inboundGroupSessionsToBackup(limit: Int): List<OlmInboundGroupSessionWrapper2>
 
     /**
      * Number of stored inbound group sessions.
@@ -404,12 +401,13 @@ internal interface IMXCryptoStore {
     fun storeUSKPrivateKey(usk: String?)
 
     fun getCrossSigningPrivateKeys(): PrivateKeysInfo?
+    fun getLiveCrossSigningPrivateKeys(): LiveData<Optional<PrivateKeysInfo>>
 
     fun saveBackupRecoveryKey(recoveryKey: String?, version: String?)
     fun getKeyBackupRecoveryKeyInfo() : SavedKeyBackupKeyInfo?
 
     fun setUserKeysAsTrusted(userId: String, trusted: Boolean = true)
-    fun setDeviceTrust(userId: String, deviceId: String, crossSignedVerified: Boolean, locallyVerified: Boolean)
+    fun setDeviceTrust(userId: String, deviceId: String, crossSignedVerified: Boolean, locallyVerified: Boolean?)
 
     fun clearOtherUserTrust()
 

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/store/db/Helper.kt

@@ -62,6 +62,7 @@ fun doRealmTransaction(realmConfiguration: RealmConfiguration, action: (Realm) -
         realm.executeTransaction { action.invoke(it) }
     }
 }
+
 fun doRealmTransactionAsync(realmConfiguration: RealmConfiguration, action: (Realm) -> Unit) {
     Realm.getInstance(realmConfiguration).use { realm ->
         realm.executeTransactionAsync { action.invoke(it) }
@@ -79,31 +80,26 @@ fun serializeForRealm(o: Any?): String? {
     val baos = ByteArrayOutputStream()
     val gzis = CompatUtil.createGzipOutputStream(baos)
     val out = ObjectOutputStream(gzis)
-
-    out.writeObject(o)
-    out.close()
-
+    out.use {
+        it.writeObject(o)
+    }
     return Base64.encodeToString(baos.toByteArray(), Base64.DEFAULT)
 }
 
 /**
  * Do the opposite of serializeForRealm.
  */
+@Suppress("UNCHECKED_CAST")
 fun <T> deserializeFromRealm(string: String?): T? {
     if (string == null) {
         return null
     }
-
     val decodedB64 = Base64.decode(string.toByteArray(), Base64.DEFAULT)
 
     val bais = ByteArrayInputStream(decodedB64)
     val gzis = GZIPInputStream(bais)
     val ois = ObjectInputStream(gzis)
-
-    @Suppress("UNCHECKED_CAST")
-    val result = ois.readObject() as T
-
-    ois.close()
-
-    return result
+    return ois.use {
+        it.readObject() as T
+    }
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/store/db/RealmCryptoStore.kt

@@ -36,16 +36,17 @@ import im.vector.matrix.android.internal.crypto.OutgoingGossipingRequestState
 import im.vector.matrix.android.internal.crypto.OutgoingRoomKeyRequest
 import im.vector.matrix.android.internal.crypto.OutgoingSecretRequest
 import im.vector.matrix.android.internal.crypto.algorithms.olm.OlmDecryptionResult
-import im.vector.matrix.android.internal.crypto.crosssigning.DeviceTrustLevel
 import im.vector.matrix.android.internal.crypto.model.CryptoCrossSigningKey
 import im.vector.matrix.android.internal.crypto.model.CryptoDeviceInfo
-import im.vector.matrix.android.internal.crypto.model.OlmInboundGroupSessionWrapper
+import im.vector.matrix.android.internal.crypto.model.OlmInboundGroupSessionWrapper2
 import im.vector.matrix.android.internal.crypto.model.OlmSessionWrapper
+import im.vector.matrix.android.internal.crypto.model.rest.DeviceInfo
 import im.vector.matrix.android.internal.crypto.model.rest.RoomKeyRequestBody
 import im.vector.matrix.android.internal.crypto.model.toEntity
 import im.vector.matrix.android.internal.crypto.store.IMXCryptoStore
 import im.vector.matrix.android.internal.crypto.store.PrivateKeysInfo
 import im.vector.matrix.android.internal.crypto.store.SavedKeyBackupKeyInfo
+import im.vector.matrix.android.internal.crypto.store.db.mapper.CrossSigningKeysMapper
 import im.vector.matrix.android.internal.crypto.store.db.model.CrossSigningInfoEntity
 import im.vector.matrix.android.internal.crypto.store.db.model.CrossSigningInfoEntityFields
 import im.vector.matrix.android.internal.crypto.store.db.model.CryptoMapper
@@ -57,8 +58,8 @@ import im.vector.matrix.android.internal.crypto.store.db.model.DeviceInfoEntityF
 import im.vector.matrix.android.internal.crypto.store.db.model.GossipingEventEntity
 import im.vector.matrix.android.internal.crypto.store.db.model.IncomingGossipingRequestEntity
 import im.vector.matrix.android.internal.crypto.store.db.model.IncomingGossipingRequestEntityFields
-import im.vector.matrix.android.internal.crypto.store.db.model.KeyInfoEntity
 import im.vector.matrix.android.internal.crypto.store.db.model.KeysBackupDataEntity
+import im.vector.matrix.android.internal.crypto.store.db.model.MyDeviceLastSeenInfoEntity
 import im.vector.matrix.android.internal.crypto.store.db.model.OlmInboundGroupSessionEntity
 import im.vector.matrix.android.internal.crypto.store.db.model.OlmInboundGroupSessionEntityFields
 import im.vector.matrix.android.internal.crypto.store.db.model.OlmSessionEntity
@@ -79,7 +80,6 @@ import im.vector.matrix.android.internal.di.MoshiProvider
 import im.vector.matrix.android.internal.session.SessionScope
 import io.realm.Realm
 import io.realm.RealmConfiguration
-import io.realm.RealmList
 import io.realm.Sort
 import io.realm.kotlin.where
 import org.matrix.olm.OlmAccount
@@ -91,6 +91,7 @@ import kotlin.collections.set
 @SessionScope
 internal class RealmCryptoStore @Inject constructor(
         @CryptoDatabase private val realmConfiguration: RealmConfiguration,
+        private val crossSigningKeysMapper: CrossSigningKeysMapper,
         private val credentials: Credentials) : IMXCryptoStore {
 
     /* ==========================================================================================
@@ -107,7 +108,7 @@ internal class RealmCryptoStore @Inject constructor(
     private val olmSessionsToRelease = HashMap<String, OlmSessionWrapper>()
 
     // Cache for InboundGroupSession, to release them properly
-    private val inboundGroupSessionToRelease = HashMap<String, OlmInboundGroupSessionWrapper>()
+    private val inboundGroupSessionToRelease = HashMap<String, OlmInboundGroupSessionWrapper2>()
 
     private val newSessionListeners = ArrayList<NewSessionListener>()
 
@@ -200,9 +201,9 @@ internal class RealmCryptoStore @Inject constructor(
     }
 
     override fun getDeviceId(): String {
-        return doRealmQueryAndCopy(realmConfiguration) {
-            it.where<CryptoMetadataEntity>().findFirst()
-        }?.deviceId ?: ""
+        return doWithRealm(realmConfiguration) {
+            it.where<CryptoMetadataEntity>().findFirst()?.deviceId
+        } ?: ""
     }
 
     override fun saveOlmAccount() {
@@ -232,48 +233,26 @@ internal class RealmCryptoStore @Inject constructor(
         return olmAccount!!
     }
 
-    override fun storeUserDevice(userId: String?, deviceInfo: CryptoDeviceInfo?) {
-        if (userId == null || deviceInfo == null) {
-            return
-        }
-
-        doRealmTransaction(realmConfiguration) { realm ->
-            val user = UserEntity.getOrCreate(realm, userId)
-
-            // Create device info
-            val deviceInfoEntity = CryptoMapper.mapToEntity(deviceInfo)
-            realm.insertOrUpdate(deviceInfoEntity)
-//            val deviceInfoEntity = DeviceInfoEntity.getOrCreate(it, userId, deviceInfo.deviceId).apply {
-//                deviceId = deviceInfo.deviceId
-//                identityKey = deviceInfo.identityKey()
-//                putDeviceInfo(deviceInfo)
-//            }
-
-            if (!user.devices.contains(deviceInfoEntity)) {
-                user.devices.add(deviceInfoEntity)
-            }
-        }
-    }
-
     override fun getUserDevice(userId: String, deviceId: String): CryptoDeviceInfo? {
-        return doRealmQueryAndCopy(realmConfiguration) {
+        return doWithRealm(realmConfiguration) {
             it.where<DeviceInfoEntity>()
                     .equalTo(DeviceInfoEntityFields.PRIMARY_KEY, DeviceInfoEntity.createPrimaryKey(userId, deviceId))
                     .findFirst()
-        }?.let {
-            CryptoMapper.mapToModel(it)
+                    ?.let { deviceInfo ->
+                        CryptoMapper.mapToModel(deviceInfo)
+                    }
         }
     }
 
     override fun deviceWithIdentityKey(identityKey: String): CryptoDeviceInfo? {
-        return doRealmQueryAndCopy(realmConfiguration) {
+        return doWithRealm(realmConfiguration) {
             it.where<DeviceInfoEntity>()
                     .equalTo(DeviceInfoEntityFields.IDENTITY_KEY, identityKey)
                     .findFirst()
+                    ?.let { deviceInfo ->
+                        CryptoMapper.mapToModel(deviceInfo)
+                    }
         }
-                ?.let {
-                    CryptoMapper.mapToModel(it)
-                }
     }
 
     override fun storeUserDevices(userId: String, devices: Map<String, CryptoDeviceInfo>?) {
@@ -285,10 +264,16 @@ internal class RealmCryptoStore @Inject constructor(
                 UserEntity.getOrCreate(realm, userId)
                         .let { u ->
                             // Add the devices
+                            val currentKnownDevices = u.devices.toList()
+                            val new = devices.map { entry -> entry.value.toEntity() }
+                            new.forEach { entity ->
+                                // Maintain first time seen
+                                val existing = currentKnownDevices.firstOrNull { it.deviceId == entity.deviceId && it.identityKey == entity.identityKey }
+                                entity.firstTimeSeenLocalTs = existing?.firstTimeSeenLocalTs ?: System.currentTimeMillis()
+                                realm.insertOrUpdate(entity)
+                            }
                             // Ensure all other devices are deleted
                             u.devices.deleteAllFromRealm()
-                            val new = devices.map { entry -> entry.value.toEntity() }
-                            new.forEach { realm.insertOrUpdate(it) }
                             u.devices.addAll(new)
                         }
             }
@@ -309,36 +294,19 @@ internal class RealmCryptoStore @Inject constructor(
                         } else {
                             CrossSigningInfoEntity.getOrCreate(realm, userId).let { signingInfo ->
                                 // What should we do if we detect a change of the keys?
-
                                 val existingMaster = signingInfo.getMasterKey()
                                 if (existingMaster != null && existingMaster.publicKeyBase64 == masterKey.unpaddedBase64PublicKey) {
-                                    // update signatures?
-                                    existingMaster.putSignatures(masterKey.signatures)
-                                    existingMaster.usages = masterKey.usages?.toTypedArray()?.let { RealmList(*it) }
-                                            ?: RealmList()
+                                    crossSigningKeysMapper.update(existingMaster, masterKey)
                                 } else {
-                                    val keyEntity = realm.createObject(KeyInfoEntity::class.java).apply {
-                                        this.publicKeyBase64 = masterKey.unpaddedBase64PublicKey
-                                        this.usages = masterKey.usages?.toTypedArray()?.let { RealmList(*it) }
-                                                ?: RealmList()
-                                        this.putSignatures(masterKey.signatures)
-                                    }
+                                    val keyEntity = crossSigningKeysMapper.map(masterKey)
                                     signingInfo.setMasterKey(keyEntity)
                                 }
 
                                 val existingSelfSigned = signingInfo.getSelfSignedKey()
                                 if (existingSelfSigned != null && existingSelfSigned.publicKeyBase64 == selfSigningKey.unpaddedBase64PublicKey) {
-                                    // update signatures?
-                                    existingSelfSigned.putSignatures(selfSigningKey.signatures)
-                                    existingSelfSigned.usages = selfSigningKey.usages?.toTypedArray()?.let { RealmList(*it) }
-                                            ?: RealmList()
+                                    crossSigningKeysMapper.update(existingSelfSigned, selfSigningKey)
                                 } else {
-                                    val keyEntity = realm.createObject(KeyInfoEntity::class.java).apply {
-                                        this.publicKeyBase64 = selfSigningKey.unpaddedBase64PublicKey
-                                        this.usages = selfSigningKey.usages?.toTypedArray()?.let { RealmList(*it) }
-                                                ?: RealmList()
-                                        this.putSignatures(selfSigningKey.signatures)
-                                    }
+                                    val keyEntity = crossSigningKeysMapper.map(selfSigningKey)
                                     signingInfo.setSelfSignedKey(keyEntity)
                                 }
 
@@ -346,21 +314,12 @@ internal class RealmCryptoStore @Inject constructor(
                                 if (userSigningKey != null) {
                                     val existingUSK = signingInfo.getUserSigningKey()
                                     if (existingUSK != null && existingUSK.publicKeyBase64 == userSigningKey.unpaddedBase64PublicKey) {
-                                        // update signatures?
-                                        existingUSK.putSignatures(userSigningKey.signatures)
-                                        existingUSK.usages = userSigningKey.usages?.toTypedArray()?.let { RealmList(*it) }
-                                                ?: RealmList()
+                                        crossSigningKeysMapper.update(existingUSK, userSigningKey)
                                     } else {
-                                        val keyEntity = realm.createObject(KeyInfoEntity::class.java).apply {
-                                            this.publicKeyBase64 = userSigningKey.unpaddedBase64PublicKey
-                                            this.usages = userSigningKey.usages?.toTypedArray()?.let { RealmList(*it) }
-                                                    ?: RealmList()
-                                            this.putSignatures(userSigningKey.signatures)
-                                        }
+                                        val keyEntity = crossSigningKeysMapper.map(userSigningKey)
                                         signingInfo.setUserSignedKey(keyEntity)
                                     }
                                 }
-
                                 userEntity.crossSigningInfoEntity = signingInfo
                             }
                         }
@@ -369,14 +328,35 @@ internal class RealmCryptoStore @Inject constructor(
     }
 
     override fun getCrossSigningPrivateKeys(): PrivateKeysInfo? {
-        return doRealmQueryAndCopy(realmConfiguration) { realm ->
-            realm.where<CryptoMetadataEntity>().findFirst()
-        }?.let {
-            PrivateKeysInfo(
-                    master = it.xSignMasterPrivateKey,
-                    selfSigned = it.xSignSelfSignedPrivateKey,
-                    user = it.xSignUserPrivateKey
-            )
+        return doWithRealm(realmConfiguration) { realm ->
+            realm.where<CryptoMetadataEntity>()
+                    .findFirst()
+                    ?.let {
+                        PrivateKeysInfo(
+                                master = it.xSignMasterPrivateKey,
+                                selfSigned = it.xSignSelfSignedPrivateKey,
+                                user = it.xSignUserPrivateKey
+                        )
+                    }
+        }
+    }
+
+    override fun getLiveCrossSigningPrivateKeys(): LiveData<Optional<PrivateKeysInfo>> {
+        val liveData = monarchy.findAllMappedWithChanges(
+                { realm: Realm ->
+                    realm
+                            .where<CryptoMetadataEntity>()
+                },
+                {
+                    PrivateKeysInfo(
+                            master = it.xSignMasterPrivateKey,
+                            selfSigned = it.xSignSelfSignedPrivateKey,
+                            user = it.xSignUserPrivateKey
+                    )
+                }
+        )
+        return Transformations.map(liveData) {
+            it.firstOrNull().toOptional()
         }
     }
 
@@ -400,16 +380,18 @@ internal class RealmCryptoStore @Inject constructor(
     }
 
     override fun getKeyBackupRecoveryKeyInfo(): SavedKeyBackupKeyInfo? {
-        return doRealmQueryAndCopy(realmConfiguration) { realm ->
-            realm.where<CryptoMetadataEntity>().findFirst()
-        }?.let {
-            val key = it.keyBackupRecoveryKey
-            val version = it.keyBackupRecoveryKeyVersion
-            if (!key.isNullOrBlank() && !version.isNullOrBlank()) {
-                SavedKeyBackupKeyInfo(recoveryKey = key, version = version)
-            } else {
-                null
-            }
+        return doWithRealm(realmConfiguration) { realm ->
+            realm.where<CryptoMetadataEntity>()
+                    .findFirst()
+                    ?.let {
+                        val key = it.keyBackupRecoveryKey
+                        val version = it.keyBackupRecoveryKeyVersion
+                        if (!key.isNullOrBlank() && !version.isNullOrBlank()) {
+                            SavedKeyBackupKeyInfo(recoveryKey = key, version = version)
+                        } else {
+                            null
+                        }
+                    }
         }
     }
 
@@ -430,24 +412,30 @@ internal class RealmCryptoStore @Inject constructor(
     }
 
     override fun getUserDevices(userId: String): Map<String, CryptoDeviceInfo>? {
-        return doRealmQueryAndCopy(realmConfiguration) {
+        return doWithRealm(realmConfiguration) {
             it.where<UserEntity>()
                     .equalTo(UserEntityFields.USER_ID, userId)
                     .findFirst()
+                    ?.devices
+                    ?.map { deviceInfo ->
+                        CryptoMapper.mapToModel(deviceInfo)
+                    }
+                    ?.associateBy { cryptoDevice ->
+                        cryptoDevice.deviceId
+                    }
         }
-                ?.devices
-                ?.map { CryptoMapper.mapToModel(it) }
-                ?.associateBy { it.deviceId }
     }
 
     override fun getUserDeviceList(userId: String): List<CryptoDeviceInfo>? {
-        return doRealmQueryAndCopy(realmConfiguration) {
+        return doWithRealm(realmConfiguration) {
             it.where<UserEntity>()
                     .equalTo(UserEntityFields.USER_ID, userId)
                     .findFirst()
+                    ?.devices
+                    ?.map { deviceInfo ->
+                        CryptoMapper.mapToModel(deviceInfo)
+                    }
         }
-                ?.devices
-                ?.map { CryptoMapper.mapToModel(it) }
     }
 
     override fun getLiveDeviceList(userId: String): LiveData<List<CryptoDeviceInfo>> {
@@ -496,6 +484,52 @@ internal class RealmCryptoStore @Inject constructor(
         }
     }
 
+    override fun getMyDevicesInfo(): List<DeviceInfo> {
+        return monarchy.fetchAllCopiedSync {
+            it.where<MyDeviceLastSeenInfoEntity>()
+        }.map {
+            DeviceInfo(
+                    deviceId = it.deviceId,
+                    lastSeenIp = it.lastSeenIp,
+                    lastSeenTs = it.lastSeenTs,
+                    displayName = it.displayName
+            )
+        }
+    }
+
+    override fun getLiveMyDevicesInfo(): LiveData<List<DeviceInfo>> {
+        return monarchy.findAllMappedWithChanges(
+                { realm: Realm ->
+                    realm.where<MyDeviceLastSeenInfoEntity>()
+                },
+                { entity ->
+                    DeviceInfo(
+                            deviceId = entity.deviceId,
+                            lastSeenIp = entity.lastSeenIp,
+                            lastSeenTs = entity.lastSeenTs,
+                            displayName = entity.displayName
+                    )
+                }
+        )
+    }
+
+    override fun saveMyDevicesInfo(info: List<DeviceInfo>) {
+        val entities = info.map {
+            MyDeviceLastSeenInfoEntity(
+                    lastSeenTs = it.lastSeenTs,
+                    lastSeenIp = it.lastSeenIp,
+                    displayName = it.displayName,
+                    deviceId = it.deviceId
+            )
+        }
+        monarchy.writeAsync { realm ->
+            realm.where<MyDeviceLastSeenInfoEntity>().findAll().deleteAllFromRealm()
+            entities.forEach {
+                realm.insertOrUpdate(it)
+            }
+        }
+    }
+
     override fun storeRoomAlgorithm(roomId: String, algorithm: String) {
         doRealmTransaction(realmConfiguration) {
             CryptoRoomEntity.getOrCreate(it, roomId).algorithm = algorithm
@@ -503,17 +537,16 @@ internal class RealmCryptoStore @Inject constructor(
     }
 
     override fun getRoomAlgorithm(roomId: String): String? {
-        return doRealmQueryAndCopy(realmConfiguration) {
-            CryptoRoomEntity.getById(it, roomId)
+        return doWithRealm(realmConfiguration) {
+            CryptoRoomEntity.getById(it, roomId)?.algorithm
         }
-                ?.algorithm
     }
 
     override fun shouldEncryptForInvitedMembers(roomId: String): Boolean {
-        return doRealmQueryAndCopy(realmConfiguration) {
-            CryptoRoomEntity.getById(it, roomId)
+        return doWithRealm(realmConfiguration) {
+            CryptoRoomEntity.getById(it, roomId)?.shouldEncryptForInvitedMembers
         }
-                ?.shouldEncryptForInvitedMembers ?: false
+                ?: false
     }
 
     override fun setShouldEncryptForInvitedMembers(roomId: String, shouldEncryptForInvitedMembers: Boolean) {
@@ -555,11 +588,7 @@ internal class RealmCryptoStore @Inject constructor(
         }
     }
 
-    override fun getDeviceSession(sessionId: String?, deviceKey: String?): OlmSessionWrapper? {
-        if (sessionId == null || deviceKey == null) {
-            return null
-        }
-
+    override fun getDeviceSession(sessionId: String, deviceKey: String): OlmSessionWrapper? {
         val key = OlmSessionEntity.createPrimaryKey(sessionId, deviceKey)
 
         // If not in cache (or not found), try to read it from realm
@@ -581,28 +610,28 @@ internal class RealmCryptoStore @Inject constructor(
     }
 
     override fun getLastUsedSessionId(deviceKey: String): String? {
-        return doRealmQueryAndCopy(realmConfiguration) {
+        return doWithRealm(realmConfiguration) {
             it.where<OlmSessionEntity>()
                     .equalTo(OlmSessionEntityFields.DEVICE_KEY, deviceKey)
                     .sort(OlmSessionEntityFields.LAST_RECEIVED_MESSAGE_TS, Sort.DESCENDING)
                     .findFirst()
+                    ?.sessionId
         }
-                ?.sessionId
     }
 
     override fun getDeviceSessionIds(deviceKey: String): MutableSet<String> {
-        return doRealmQueryAndCopyList(realmConfiguration) {
+        return doWithRealm(realmConfiguration) {
             it.where<OlmSessionEntity>()
                     .equalTo(OlmSessionEntityFields.DEVICE_KEY, deviceKey)
                     .findAll()
+                    .mapNotNull { sessionEntity ->
+                        sessionEntity.sessionId
+                    }
         }
-                .mapNotNull {
-                    it.sessionId
-                }
                 .toMutableSet()
     }
 
-    override fun storeInboundGroupSessions(sessions: List<OlmInboundGroupSessionWrapper>) {
+    override fun storeInboundGroupSessions(sessions: List<OlmInboundGroupSessionWrapper2>) {
         if (sessions.isEmpty()) {
             return
         }
@@ -640,17 +669,17 @@ internal class RealmCryptoStore @Inject constructor(
         }
     }
 
-    override fun getInboundGroupSession(sessionId: String, senderKey: String): OlmInboundGroupSessionWrapper? {
+    override fun getInboundGroupSession(sessionId: String, senderKey: String): OlmInboundGroupSessionWrapper2? {
         val key = OlmInboundGroupSessionEntity.createPrimaryKey(sessionId, senderKey)
 
         // If not in cache (or not found), try to read it from realm
         if (inboundGroupSessionToRelease[key] == null) {
-            doRealmQueryAndCopy(realmConfiguration) {
+            doWithRealm(realmConfiguration) {
                 it.where<OlmInboundGroupSessionEntity>()
                         .equalTo(OlmInboundGroupSessionEntityFields.PRIMARY_KEY, key)
                         .findFirst()
+                        ?.getInboundGroupSession()
             }
-                    ?.getInboundGroupSession()
                     ?.let {
                         inboundGroupSessionToRelease[key] = it
                     }
@@ -660,17 +689,17 @@ internal class RealmCryptoStore @Inject constructor(
     }
 
     /**
-     * Note: the result will be only use to export all the keys and not to use the OlmInboundGroupSessionWrapper,
+     * Note: the result will be only use to export all the keys and not to use the OlmInboundGroupSessionWrapper2,
      * so there is no need to use or update `inboundGroupSessionToRelease` for native memory management
      */
-    override fun getInboundGroupSessions(): MutableList<OlmInboundGroupSessionWrapper> {
-        return doRealmQueryAndCopyList(realmConfiguration) {
+    override fun getInboundGroupSessions(): MutableList<OlmInboundGroupSessionWrapper2> {
+        return doWithRealm(realmConfiguration) {
             it.where<OlmInboundGroupSessionEntity>()
                     .findAll()
+                    .mapNotNull { inboundGroupSessionEntity ->
+                        inboundGroupSessionEntity.getInboundGroupSession()
+                    }
         }
-                .mapNotNull {
-                    it.getInboundGroupSession()
-                }
                 .toMutableList()
     }
 
@@ -735,7 +764,7 @@ internal class RealmCryptoStore @Inject constructor(
         }
     }
 
-    override fun markBackupDoneForInboundGroupSessions(olmInboundGroupSessionWrappers: List<OlmInboundGroupSessionWrapper>) {
+    override fun markBackupDoneForInboundGroupSessions(olmInboundGroupSessionWrappers: List<OlmInboundGroupSessionWrapper2>) {
         if (olmInboundGroupSessionWrappers.isEmpty()) {
             return
         }
@@ -758,14 +787,15 @@ internal class RealmCryptoStore @Inject constructor(
         }
     }
 
-    override fun inboundGroupSessionsToBackup(limit: Int): List<OlmInboundGroupSessionWrapper> {
-        return doRealmQueryAndCopyList(realmConfiguration) {
+    override fun inboundGroupSessionsToBackup(limit: Int): List<OlmInboundGroupSessionWrapper2> {
+        return doWithRealm(realmConfiguration) {
             it.where<OlmInboundGroupSessionEntity>()
                     .equalTo(OlmInboundGroupSessionEntityFields.BACKED_UP, false)
                     .limit(limit.toLong())
                     .findAll()
-        }.mapNotNull { inboundGroupSession ->
-            inboundGroupSession.getInboundGroupSession()
+                    .mapNotNull { inboundGroupSession ->
+                        inboundGroupSession.getInboundGroupSession()
+                    }
         }
     }
 
@@ -789,10 +819,9 @@ internal class RealmCryptoStore @Inject constructor(
     }
 
     override fun getGlobalBlacklistUnverifiedDevices(): Boolean {
-        return doRealmQueryAndCopy(realmConfiguration) {
-            it.where<CryptoMetadataEntity>().findFirst()
-        }?.globalBlacklistUnverifiedDevices
-                ?: false
+        return doWithRealm(realmConfiguration) {
+            it.where<CryptoMetadataEntity>().findFirst()?.globalBlacklistUnverifiedDevices
+        } ?: false
     }
 
     override fun setRoomsListBlacklistUnverifiedDevices(roomIds: List<String>) {
@@ -815,28 +844,28 @@ internal class RealmCryptoStore @Inject constructor(
     }
 
     override fun getRoomsListBlacklistUnverifiedDevices(): MutableList<String> {
-        return doRealmQueryAndCopyList(realmConfiguration) {
+        return doWithRealm(realmConfiguration) {
             it.where<CryptoRoomEntity>()
                     .equalTo(CryptoRoomEntityFields.BLACKLIST_UNVERIFIED_DEVICES, true)
                     .findAll()
+                    .mapNotNull { cryptoRoom ->
+                        cryptoRoom.roomId
+                    }
         }
-                .mapNotNull {
-                    it.roomId
-                }
                 .toMutableList()
     }
 
     override fun getDeviceTrackingStatuses(): MutableMap<String, Int> {
-        return doRealmQueryAndCopyList(realmConfiguration) {
+        return doWithRealm(realmConfiguration) {
             it.where<UserEntity>()
                     .findAll()
+                    .associateBy { user ->
+                        user.userId!!
+                    }
+                    .mapValues { entry ->
+                        entry.value.deviceTrackingStatus
+                    }
         }
-                .associateBy {
-                    it.userId!!
-                }
-                .mapValues {
-                    it.value.deviceTrackingStatus
-                }
                 .toMutableMap()
     }
 
@@ -851,12 +880,12 @@ internal class RealmCryptoStore @Inject constructor(
     }
 
     override fun getDeviceTrackingStatus(userId: String, defaultValue: Int): Int {
-        return doRealmQueryAndCopy(realmConfiguration) {
+        return doWithRealm(realmConfiguration) {
             it.where<UserEntity>()
                     .equalTo(UserEntityFields.USER_ID, userId)
                     .findFirst()
+                    ?.deviceTrackingStatus
         }
-                ?.deviceTrackingStatus
                 ?: defaultValue
     }
 
@@ -1093,63 +1122,65 @@ internal class RealmCryptoStore @Inject constructor(
     }
 
     override fun getIncomingRoomKeyRequest(userId: String, deviceId: String, requestId: String): IncomingRoomKeyRequest? {
-        return doRealmQueryAndCopyList(realmConfiguration) { realm ->
+        return doWithRealm(realmConfiguration) { realm ->
             realm.where<IncomingGossipingRequestEntity>()
                     .equalTo(IncomingGossipingRequestEntityFields.TYPE_STR, GossipRequestType.KEY.name)
                     .equalTo(IncomingGossipingRequestEntityFields.OTHER_DEVICE_ID, deviceId)
                     .equalTo(IncomingGossipingRequestEntityFields.OTHER_USER_ID, userId)
                     .findAll()
-        }.mapNotNull { entity ->
-            entity.toIncomingGossipingRequest() as? IncomingRoomKeyRequest
-        }.firstOrNull()
+                    .mapNotNull { entity ->
+                        entity.toIncomingGossipingRequest() as? IncomingRoomKeyRequest
+                    }
+                    .firstOrNull()
+        }
     }
 
     override fun getPendingIncomingRoomKeyRequests(): List<IncomingRoomKeyRequest> {
-        return doRealmQueryAndCopyList(realmConfiguration) {
+        return doWithRealm(realmConfiguration) {
             it.where<IncomingGossipingRequestEntity>()
                     .equalTo(IncomingGossipingRequestEntityFields.TYPE_STR, GossipRequestType.KEY.name)
                     .equalTo(IncomingGossipingRequestEntityFields.REQUEST_STATE_STR, GossipingRequestState.PENDING.name)
                     .findAll()
+                    .map { entity ->
+                        IncomingRoomKeyRequest(
+                                userId = entity.otherUserId,
+                                deviceId = entity.otherDeviceId,
+                                requestId = entity.requestId,
+                                requestBody = entity.getRequestedKeyInfo(),
+                                localCreationTimestamp = entity.localCreationTimestamp
+                        )
+                    }
         }
-                .map { entity ->
-                    IncomingRoomKeyRequest(
-                            userId = entity.otherUserId,
-                            deviceId = entity.otherDeviceId,
-                            requestId = entity.requestId,
-                            requestBody = entity.getRequestedKeyInfo(),
-                            localCreationTimestamp = entity.localCreationTimestamp
-                    )
-                }
     }
 
     override fun getPendingIncomingGossipingRequests(): List<IncomingShareRequestCommon> {
-        return doRealmQueryAndCopyList(realmConfiguration) {
+        return doWithRealm(realmConfiguration) {
             it.where<IncomingGossipingRequestEntity>()
                     .equalTo(IncomingGossipingRequestEntityFields.REQUEST_STATE_STR, GossipingRequestState.PENDING.name)
                     .findAll()
-        }
-                .mapNotNull { entity ->
-                    when (entity.type) {
-                        GossipRequestType.KEY    -> {
-                            IncomingRoomKeyRequest(
-                                    userId = entity.otherUserId,
-                                    deviceId = entity.otherDeviceId,
-                                    requestId = entity.requestId,
-                                    requestBody = entity.getRequestedKeyInfo(),
-                                    localCreationTimestamp = entity.localCreationTimestamp
-                            )
-                        }
-                        GossipRequestType.SECRET -> {
-                            IncomingSecretShareRequest(
-                                    userId = entity.otherUserId,
-                                    deviceId = entity.otherDeviceId,
-                                    requestId = entity.requestId,
-                                    secretName = entity.getRequestedSecretName(),
-                                    localCreationTimestamp = entity.localCreationTimestamp
-                            )
+                    .mapNotNull { entity ->
+                        when (entity.type) {
+                            GossipRequestType.KEY    -> {
+                                IncomingRoomKeyRequest(
+                                        userId = entity.otherUserId,
+                                        deviceId = entity.otherDeviceId,
+                                        requestId = entity.requestId,
+                                        requestBody = entity.getRequestedKeyInfo(),
+                                        localCreationTimestamp = entity.localCreationTimestamp
+                                )
+                            }
+                            GossipRequestType.SECRET -> {
+                                IncomingSecretShareRequest(
+                                        userId = entity.otherUserId,
+                                        deviceId = entity.otherDeviceId,
+                                        requestId = entity.requestId,
+                                        secretName = entity.getRequestedSecretName(),
+                                        localCreationTimestamp = entity.localCreationTimestamp
+                                )
+                            }
                         }
                     }
-                }
+        }
     }
 
     override fun storeIncomingGossipingRequest(request: IncomingShareRequestCommon, ageLocalTS: Long?) {
@@ -1187,9 +1218,9 @@ internal class RealmCryptoStore @Inject constructor(
      * Cross Signing
      * ========================================================================================== */
     override fun getMyCrossSigningInfo(): MXCrossSigningInfo? {
-        return doRealmQueryAndCopy(realmConfiguration) {
-            it.where<CryptoMetadataEntity>().findFirst()
-        }?.userId?.let {
+        return doWithRealm(realmConfiguration) {
+            it.where<CryptoMetadataEntity>().findFirst()?.userId
+        }?.let {
             getCrossSigningInfo(it)
         }
     }
@@ -1222,7 +1253,7 @@ internal class RealmCryptoStore @Inject constructor(
         }
     }
 
-    override fun setDeviceTrust(userId: String, deviceId: String, crossSignedVerified: Boolean, locallyVerified: Boolean) {
+    override fun setDeviceTrust(userId: String, deviceId: String, crossSignedVerified: Boolean, locallyVerified: Boolean?) {
         doRealmTransaction(realmConfiguration) { realm ->
             realm.where(DeviceInfoEntity::class.java)
                     .equalTo(DeviceInfoEntityFields.PRIMARY_KEY, DeviceInfoEntity.createPrimaryKey(userId, deviceId))
@@ -1235,7 +1266,7 @@ internal class RealmCryptoStore @Inject constructor(
                                 deviceInfoEntity.trustLevelEntity = it
                             }
                         } else {
-                            trustEntity.locallyVerified = locallyVerified
+                            locallyVerified?.let { trustEntity.locallyVerified = it  }
                             trustEntity.crossSignedVerified = crossSignedVerified
                         }
                     }
@@ -1308,33 +1339,24 @@ internal class RealmCryptoStore @Inject constructor(
     }
 
     override fun getCrossSigningInfo(userId: String): MXCrossSigningInfo? {
-        return doRealmQueryAndCopy(realmConfiguration) { realm ->
-            realm.where(CrossSigningInfoEntity::class.java)
+        return doWithRealm(realmConfiguration) { realm ->
+            val crossSigningInfo = realm.where(CrossSigningInfoEntity::class.java)
                     .equalTo(CrossSigningInfoEntityFields.USER_ID, userId)
                     .findFirst()
-        }?.let { xsignInfo ->
-            mapCrossSigningInfoEntity(xsignInfo)
+            if (crossSigningInfo == null) {
+                null
+            } else {
+                mapCrossSigningInfoEntity(crossSigningInfo)
+            }
         }
     }
 
     private fun mapCrossSigningInfoEntity(xsignInfo: CrossSigningInfoEntity): MXCrossSigningInfo {
+        val userId = xsignInfo.userId ?: ""
         return MXCrossSigningInfo(
-                userId = xsignInfo.userId ?: "",
+                userId = userId,
                 crossSigningKeys = xsignInfo.crossSigningKeys.mapNotNull {
-                    val pubKey = it.publicKeyBase64 ?: return@mapNotNull null
-                    CryptoCrossSigningKey(
-                            userId = xsignInfo.userId ?: "",
-                            keys = mapOf("ed25519:$pubKey" to pubKey),
-                            usages = it.usages.map { it },
-                            signatures = it.getSignatures(),
-                            trustLevel = it.trustLevelEntity?.let {
-                                DeviceTrustLevel(
-                                        crossSigningVerified = it.crossSignedVerified ?: false,
-                                        locallyVerified = it.locallyVerified ?: false
-                                )
-                            }
-
-                    )
+                    crossSigningKeysMapper.map(userId, it)
                 }
         )
     }
@@ -1345,26 +1367,7 @@ internal class RealmCryptoStore @Inject constructor(
                     realm.where<CrossSigningInfoEntity>()
                             .equalTo(UserEntityFields.USER_ID, userId)
                 },
-                { entity ->
-                    MXCrossSigningInfo(
-                            userId = userId,
-                            crossSigningKeys = entity.crossSigningKeys.mapNotNull {
-                                val pubKey = it.publicKeyBase64 ?: return@mapNotNull null
-                                CryptoCrossSigningKey(
-                                        userId = userId,
-                                        keys = mapOf("ed25519:$pubKey" to pubKey),
-                                        usages = it.usages.map { it },
-                                        signatures = it.getSignatures(),
-                                        trustLevel = it.trustLevelEntity?.let {
-                                            DeviceTrustLevel(
-                                                    crossSigningVerified = it.crossSignedVerified ?: false,
-                                                    locallyVerified = it.locallyVerified ?: false
-                                            )
-                                        }
-                                )
-                            }
-                    )
-                }
+                { mapCrossSigningInfoEntity(it) }
         )
         return Transformations.map(liveData) {
             it.firstOrNull().toOptional()
@@ -1395,31 +1398,21 @@ internal class RealmCryptoStore @Inject constructor(
     }
 
     private fun addOrUpdateCrossSigningInfo(realm: Realm, userId: String, info: MXCrossSigningInfo?): CrossSigningInfoEntity? {
-        var existing = CrossSigningInfoEntity.get(realm, userId)
         if (info == null) {
             // Delete known if needed
-            existing?.deleteFromRealm()
+            CrossSigningInfoEntity.get(realm, userId)?.deleteFromRealm()
+            return null
             // TODO notify, we might need to untrust things?
         } else {
             // Just override existing, caller should check and untrust id needed
-            existing = CrossSigningInfoEntity.getOrCreate(realm, userId)
-            // existing.crossSigningKeys.forEach { it.deleteFromRealm() }
-            val xkeys = RealmList<KeyInfoEntity>()
-            info.crossSigningKeys.forEach { cryptoCrossSigningKey ->
-                xkeys.add(
-                        realm.createObject(KeyInfoEntity::class.java).also { keyInfoEntity ->
-                            keyInfoEntity.publicKeyBase64 = cryptoCrossSigningKey.unpaddedBase64PublicKey
-                            keyInfoEntity.usages = cryptoCrossSigningKey.usages?.let { RealmList(*it.toTypedArray()) }
-                                    ?: RealmList()
-                            keyInfoEntity.putSignatures(cryptoCrossSigningKey.signatures)
-                            // TODO how to handle better, check if same keys?
-                            // reset trust
-                            keyInfoEntity.trustLevelEntity = null
-                        }
-                )
-            }
-            existing.crossSigningKeys = xkeys
+            val existing = CrossSigningInfoEntity.getOrCreate(realm, userId)
+            existing.crossSigningKeys.deleteAllFromRealm()
+            existing.crossSigningKeys.addAll(
+                    info.crossSigningKeys.map {
+                        crossSigningKeysMapper.map(it)
+                    }
+            )
+            return existing
         }
-        return existing
     }
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/store/db/RealmCryptoStoreMigration.kt

@@ -18,14 +18,20 @@ package im.vector.matrix.android.internal.crypto.store.db
 
 import com.squareup.moshi.Moshi
 import com.squareup.moshi.Types
+import im.vector.matrix.android.api.extensions.tryThis
 import im.vector.matrix.android.api.util.JsonDict
 import im.vector.matrix.android.internal.crypto.model.MXDeviceInfo
+import im.vector.matrix.android.internal.crypto.model.OlmInboundGroupSessionWrapper
+import im.vector.matrix.android.internal.crypto.model.OlmInboundGroupSessionWrapper2
+import im.vector.matrix.android.internal.crypto.store.db.mapper.CrossSigningKeysMapper
 import im.vector.matrix.android.internal.crypto.store.db.model.CrossSigningInfoEntityFields
 import im.vector.matrix.android.internal.crypto.store.db.model.CryptoMetadataEntityFields
 import im.vector.matrix.android.internal.crypto.store.db.model.DeviceInfoEntityFields
 import im.vector.matrix.android.internal.crypto.store.db.model.GossipingEventEntityFields
 import im.vector.matrix.android.internal.crypto.store.db.model.IncomingGossipingRequestEntityFields
 import im.vector.matrix.android.internal.crypto.store.db.model.KeyInfoEntityFields
+import im.vector.matrix.android.internal.crypto.store.db.model.MyDeviceLastSeenInfoEntityFields
+import im.vector.matrix.android.internal.crypto.store.db.model.OlmInboundGroupSessionEntityFields
 import im.vector.matrix.android.internal.crypto.store.db.model.OutgoingGossipingRequestEntityFields
 import im.vector.matrix.android.internal.crypto.store.db.model.TrustLevelEntityFields
 import im.vector.matrix.android.internal.crypto.store.db.model.UserEntityFields
@@ -33,11 +39,14 @@ import im.vector.matrix.android.internal.di.SerializeNulls
 import io.realm.DynamicRealm
 import io.realm.RealmMigration
 import timber.log.Timber
+import javax.inject.Inject
 
-internal object RealmCryptoStoreMigration : RealmMigration {
+internal class RealmCryptoStoreMigration @Inject constructor(private val crossSigningKeysMapper: CrossSigningKeysMapper) : RealmMigration {
 
     // Version 1L added Cross Signing info persistence
-    const val CRYPTO_STORE_SCHEMA_VERSION = 3L
+    companion object {
+        const val CRYPTO_STORE_SCHEMA_VERSION = 6L
+    }
 
     override fun migrate(realm: DynamicRealm, oldVersion: Long, newVersion: Long) {
         Timber.v("Migrating Realm Crypto from $oldVersion to $newVersion")
@@ -45,6 +54,9 @@ internal object RealmCryptoStoreMigration : RealmMigration {
         if (oldVersion <= 0) migrateTo1(realm)
         if (oldVersion <= 1) migrateTo2(realm)
         if (oldVersion <= 2) migrateTo3(realm)
+        if (oldVersion <= 3) migrateTo4(realm)
+        if (oldVersion <= 4) migrateTo5(realm)
+        if (oldVersion <= 5) migrateTo6(realm)
     }
 
     private fun migrateTo1(realm: DynamicRealm) {
@@ -193,4 +205,73 @@ internal object RealmCryptoStoreMigration : RealmMigration {
                 ?.addField(CryptoMetadataEntityFields.KEY_BACKUP_RECOVERY_KEY, String::class.java)
                 ?.addField(CryptoMetadataEntityFields.KEY_BACKUP_RECOVERY_KEY_VERSION, String::class.java)
     }
+
+    private fun migrateTo4(realm: DynamicRealm) {
+        Timber.d("Updating KeyInfoEntity table")
+        val keyInfoEntities = realm.where("KeyInfoEntity").findAll()
+        try {
+            keyInfoEntities.forEach {
+                val stringSignatures = it.getString(KeyInfoEntityFields.SIGNATURES)
+                val objectSignatures: Map<String, Map<String, String>>? = deserializeFromRealm(stringSignatures)
+                val jsonSignatures = crossSigningKeysMapper.serializeSignatures(objectSignatures)
+                it.setString(KeyInfoEntityFields.SIGNATURES, jsonSignatures)
+            }
+        } catch (failure: Throwable) {
+        }
+
+        // Migrate frozen classes
+        val inboundGroupSessions = realm.where("OlmInboundGroupSessionEntity").findAll()
+        inboundGroupSessions.forEach { dynamicObject ->
+            dynamicObject.getString(OlmInboundGroupSessionEntityFields.OLM_INBOUND_GROUP_SESSION_DATA)?.let { serializedObject ->
+                try {
+                    deserializeFromRealm<OlmInboundGroupSessionWrapper?>(serializedObject)?.let { oldFormat ->
+                        val newFormat = oldFormat.exportKeys()?.let {
+                            OlmInboundGroupSessionWrapper2(it)
+                        }
+                        dynamicObject.setString(OlmInboundGroupSessionEntityFields.OLM_INBOUND_GROUP_SESSION_DATA, serializeForRealm(newFormat))
+                    }
+                } catch (failure: Throwable) {
+                    Timber.e(failure, "## OlmInboundGroupSessionEntity migration failed")
+                }
+            }
+        }
+    }
+
+    private fun migrateTo5(realm: DynamicRealm) {
+        realm.schema.create("MyDeviceLastSeenInfoEntity")
+                .addField(MyDeviceLastSeenInfoEntityFields.DEVICE_ID, String::class.java)
+                .addPrimaryKey(MyDeviceLastSeenInfoEntityFields.DEVICE_ID)
+                .addField(MyDeviceLastSeenInfoEntityFields.DISPLAY_NAME, String::class.java)
+                .addField(MyDeviceLastSeenInfoEntityFields.LAST_SEEN_IP, String::class.java)
+                .addField(MyDeviceLastSeenInfoEntityFields.LAST_SEEN_TS, Long::class.java)
+                .setNullable(MyDeviceLastSeenInfoEntityFields.LAST_SEEN_TS, true)
+
+        val now = System.currentTimeMillis()
+        realm.schema.get("DeviceInfoEntity")
+                ?.addField(DeviceInfoEntityFields.FIRST_TIME_SEEN_LOCAL_TS, Long::class.java)
+                ?.setNullable(DeviceInfoEntityFields.FIRST_TIME_SEEN_LOCAL_TS, true)
+                ?.transform { deviceInfoEntity ->
+                    tryThis {
+                        deviceInfoEntity.setLong(DeviceInfoEntityFields.FIRST_TIME_SEEN_LOCAL_TS, now)
+                    }
+                }
+    }
+
+    // Fixes duplicate devices in UserEntity#devices
+    private fun migrateTo6(realm: DynamicRealm) {
+        val userEntities = realm.where("UserEntity").findAll()
+        userEntities.forEach {
+            try {
+                val deviceList = it.getList(UserEntityFields.DEVICES.`$`)
+                        ?: return@forEach
+                val distinct = deviceList.distinctBy { it.getString(DeviceInfoEntityFields.DEVICE_ID) }
+                if (distinct.size != deviceList.size) {
+                    deviceList.clear()
+                    deviceList.addAll(distinct)
+                }
+            } catch (failure: Throwable) {
+                Timber.w(failure, "Crypto Data base migration error for migrateTo6")
+            }
+        }
+    }
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/store/db/RealmCryptoStoreModule.kt

@@ -24,6 +24,7 @@ import im.vector.matrix.android.internal.crypto.store.db.model.GossipingEventEnt
 import im.vector.matrix.android.internal.crypto.store.db.model.IncomingGossipingRequestEntity
 import im.vector.matrix.android.internal.crypto.store.db.model.KeyInfoEntity
 import im.vector.matrix.android.internal.crypto.store.db.model.KeysBackupDataEntity
+import im.vector.matrix.android.internal.crypto.store.db.model.MyDeviceLastSeenInfoEntity
 import im.vector.matrix.android.internal.crypto.store.db.model.OlmInboundGroupSessionEntity
 import im.vector.matrix.android.internal.crypto.store.db.model.OlmSessionEntity
 import im.vector.matrix.android.internal.crypto.store.db.model.OutgoingGossipingRequestEntity
@@ -48,6 +49,7 @@ import io.realm.annotations.RealmModule
             TrustLevelEntity::class,
             GossipingEventEntity::class,
             IncomingGossipingRequestEntity::class,
-            OutgoingGossipingRequestEntity::class
+            OutgoingGossipingRequestEntity::class,
+            MyDeviceLastSeenInfoEntity::class
         ])
 internal class RealmCryptoStoreModule

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/store/db/mapper/CrossSigningKeysMapper.kt

@@ -0,0 +1,85 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package im.vector.matrix.android.internal.crypto.store.db.mapper
+
+import com.squareup.moshi.Moshi
+import com.squareup.moshi.Types
+import im.vector.matrix.android.internal.crypto.crosssigning.DeviceTrustLevel
+import im.vector.matrix.android.internal.crypto.model.CryptoCrossSigningKey
+import im.vector.matrix.android.internal.crypto.store.db.model.KeyInfoEntity
+import io.realm.RealmList
+import timber.log.Timber
+import javax.inject.Inject
+
+internal class CrossSigningKeysMapper @Inject constructor(moshi: Moshi) {
+
+    private val signaturesAdapter = moshi.adapter<Map<String, Map<String, String>>>(Types.newParameterizedType(
+            Map::class.java,
+            String::class.java,
+            Any::class.java
+    ))
+
+    fun update(keyInfo: KeyInfoEntity, cryptoCrossSigningKey: CryptoCrossSigningKey) {
+        // update signatures?
+        keyInfo.signatures = serializeSignatures(cryptoCrossSigningKey.signatures)
+        keyInfo.usages = cryptoCrossSigningKey.usages?.toTypedArray()?.let { RealmList(*it) }
+                ?: RealmList()
+    }
+
+    fun map(userId: String?, keyInfo: KeyInfoEntity?): CryptoCrossSigningKey? {
+        val pubKey = keyInfo?.publicKeyBase64 ?: return null
+        return CryptoCrossSigningKey(
+                userId = userId ?: "",
+                keys = mapOf("ed25519:$pubKey" to pubKey),
+                usages = keyInfo.usages.map { it },
+                signatures = deserializeSignatures(keyInfo.signatures),
+                trustLevel = keyInfo.trustLevelEntity?.let {
+                    DeviceTrustLevel(
+                            crossSigningVerified = it.crossSignedVerified ?: false,
+                            locallyVerified = it.locallyVerified ?: false
+                    )
+                }
+        )
+    }
+
+    fun map(keyInfo: CryptoCrossSigningKey): KeyInfoEntity {
+        return KeyInfoEntity().apply {
+            publicKeyBase64 = keyInfo.unpaddedBase64PublicKey
+            usages = keyInfo.usages?.let { RealmList(*it.toTypedArray()) } ?: RealmList()
+            signatures = serializeSignatures(keyInfo.signatures)
+            // TODO how to handle better, check if same keys?
+            // reset trust
+            trustLevelEntity = null
+        }
+    }
+
+    fun serializeSignatures(signatures: Map<String, Map<String, String>>?): String {
+        return signaturesAdapter.toJson(signatures)
+    }
+
+    fun deserializeSignatures(signatures: String?): Map<String, Map<String, String>>? {
+        if (signatures == null) {
+            return null
+        }
+        return try {
+            signaturesAdapter.fromJson(signatures)
+        } catch (failure: Throwable) {
+            Timber.e(failure)
+            null
+        }
+    }
+}

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/store/db/model/CryptoMapper.kt

@@ -104,7 +104,8 @@ object CryptoMapper {
                         Timber.e(failure)
                         null
                     }
-                }
+                },
+                firstTimeSeenLocalTs = deviceInfoEntity.firstTimeSeenLocalTs
         )
     }
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/store/db/model/DeviceInfoEntity.kt

@@ -34,7 +34,12 @@ internal open class DeviceInfoEntity(@PrimaryKey var primaryKey: String = "",
                                      var keysMapJson: String? = null,
                                      var signatureMapJson: String? = null,
                                      var unsignedMapJson: String? = null,
-                                     var trustLevelEntity: TrustLevelEntity? = null
+                                     var trustLevelEntity: TrustLevelEntity? = null,
+                                     /**
+                                      * We use that to make distinction between old devices (there before mine)
+                                      * and new ones. Used for example to detect new unverified login
+                                      */
+                                     var firstTimeSeenLocalTs: Long? = null
 ) : RealmObject() {
 
 //    // Deserialize data

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/store/db/model/KeyInfoEntity.kt

@@ -16,8 +16,6 @@
 
 package im.vector.matrix.android.internal.crypto.store.db.model
 
-import im.vector.matrix.android.internal.crypto.store.db.deserializeFromRealm
-import im.vector.matrix.android.internal.crypto.store.db.serializeForRealm
 import io.realm.RealmList
 import io.realm.RealmObject
 
@@ -31,15 +29,4 @@ internal open class KeyInfoEntity(
          */
         var signatures: String? = null,
         var trustLevelEntity: TrustLevelEntity? = null
-) : RealmObject() {
-
-    // Deserialize data
-    fun getSignatures(): Map<String, Map<String, String>>? {
-        return deserializeFromRealm(signatures)
-    }
-
-    // Serialize data
-    fun putSignatures(deviceInfo: Map<String, Map<String, String>>?) {
-        signatures = serializeForRealm(deviceInfo)
-    }
-}
+) : RealmObject()

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/store/db/model/MyDeviceLastSeenInfoEntity.kt

@@ -0,0 +1,34 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package im.vector.matrix.android.internal.crypto.store.db.model
+
+import io.realm.RealmObject
+import io.realm.annotations.PrimaryKey
+
+internal open class MyDeviceLastSeenInfoEntity(
+        /**The device id*/
+        @PrimaryKey var deviceId: String? = null,
+        /** The device display name*/
+        var displayName: String? = null,
+        /** The last time this device has been seen. */
+        var lastSeenTs: Long? = null,
+        /** The last ip address*/
+        var lastSeenIp: String? = null
+) : RealmObject() {
+
+    companion object
+}

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/store/db/model/OlmInboundGroupSessionEntity.kt

@@ -16,11 +16,12 @@
 
 package im.vector.matrix.android.internal.crypto.store.db.model
 
-import im.vector.matrix.android.internal.crypto.model.OlmInboundGroupSessionWrapper
+import im.vector.matrix.android.internal.crypto.model.OlmInboundGroupSessionWrapper2
 import im.vector.matrix.android.internal.crypto.store.db.deserializeFromRealm
 import im.vector.matrix.android.internal.crypto.store.db.serializeForRealm
 import io.realm.RealmObject
 import io.realm.annotations.PrimaryKey
+import timber.log.Timber
 
 internal fun OlmInboundGroupSessionEntity.Companion.createPrimaryKey(sessionId: String?, senderKey: String?) = "$sessionId|$senderKey"
 
@@ -35,11 +36,16 @@ internal open class OlmInboundGroupSessionEntity(
         var backedUp: Boolean = false)
     : RealmObject() {
 
-    fun getInboundGroupSession(): OlmInboundGroupSessionWrapper? {
-        return deserializeFromRealm(olmInboundGroupSessionData)
+    fun getInboundGroupSession(): OlmInboundGroupSessionWrapper2? {
+        return try {
+            deserializeFromRealm<OlmInboundGroupSessionWrapper2?>(olmInboundGroupSessionData)
+        } catch (failure: Throwable) {
+            Timber.e(failure, "## Deserialization failure")
+            return null
+        }
     }
 
-    fun putInboundGroupSession(olmInboundGroupSessionWrapper: OlmInboundGroupSessionWrapper?) {
+    fun putInboundGroupSession(olmInboundGroupSessionWrapper: OlmInboundGroupSessionWrapper2?) {
         olmInboundGroupSessionData = serializeForRealm(olmInboundGroupSessionWrapper)
     }
 

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/tasks/DeleteDeviceTask.kt

@@ -17,10 +17,9 @@
 package im.vector.matrix.android.internal.crypto.tasks
 
 import im.vector.matrix.android.api.failure.Failure
-import im.vector.matrix.android.internal.auth.registration.RegistrationFlowResponse
+import im.vector.matrix.android.api.failure.toRegistrationFlowResponse
 import im.vector.matrix.android.internal.crypto.api.CryptoApi
 import im.vector.matrix.android.internal.crypto.model.rest.DeleteDeviceParams
-import im.vector.matrix.android.internal.di.MoshiProvider
 import im.vector.matrix.android.internal.network.executeRequest
 import im.vector.matrix.android.internal.task.Task
 import org.greenrobot.eventbus.EventBus
@@ -43,25 +42,9 @@ internal class DefaultDeleteDeviceTask @Inject constructor(
                 apiCall = cryptoApi.deleteDevice(params.deviceId, DeleteDeviceParams())
             }
         } catch (throwable: Throwable) {
-            if (throwable is Failure.OtherServerError && throwable.httpCode == 401) {
-                // Parse to get a RegistrationFlowResponse
-                val registrationFlowResponse = try {
-                    MoshiProvider.providesMoshi()
-                            .adapter(RegistrationFlowResponse::class.java)
-                            .fromJson(throwable.errorBody)
-                } catch (e: Exception) {
-                    null
-                }
-                // check if the server response can be casted
-                if (registrationFlowResponse != null) {
-                    throw Failure.RegistrationFlowError(registrationFlowResponse)
-                } else {
-                    throw throwable
-                }
-            } else {
-                // Other error
-                throw throwable
-            }
+            throw throwable.toRegistrationFlowResponse()
+                    ?.let { Failure.RegistrationFlowError(it) }
+                    ?: throwable
         }
     }
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/tasks/SendToDeviceTask.kt

@@ -52,6 +52,8 @@ internal class DefaultSendToDeviceTask @Inject constructor(
                     params.transactionId ?: Random.nextInt(Integer.MAX_VALUE).toString(),
                     sendToDeviceBody
             )
+            isRetryable = true
+            maxRetryCount = 3
         }
     }
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/tasks/UploadSigningKeysTask.kt

@@ -17,14 +17,13 @@
 package im.vector.matrix.android.internal.crypto.tasks
 
 import im.vector.matrix.android.api.failure.Failure
-import im.vector.matrix.android.internal.auth.registration.RegistrationFlowResponse
+import im.vector.matrix.android.api.failure.toRegistrationFlowResponse
 import im.vector.matrix.android.internal.crypto.api.CryptoApi
 import im.vector.matrix.android.internal.crypto.model.CryptoCrossSigningKey
 import im.vector.matrix.android.internal.crypto.model.rest.KeysQueryResponse
 import im.vector.matrix.android.internal.crypto.model.rest.UploadSigningKeysBody
 import im.vector.matrix.android.internal.crypto.model.rest.UserPasswordAuth
 import im.vector.matrix.android.internal.crypto.model.toRest
-import im.vector.matrix.android.internal.di.MoshiProvider
 import im.vector.matrix.android.internal.network.executeRequest
 import im.vector.matrix.android.internal.task.Task
 import org.greenrobot.eventbus.EventBus
@@ -65,37 +64,25 @@ internal class DefaultUploadSigningKeysTask @Inject constructor(
             }
             return
         } catch (throwable: Throwable) {
-            if (throwable is Failure.OtherServerError
-                    && throwable.httpCode == 401
+            val registrationFlowResponse = throwable.toRegistrationFlowResponse()
+            if (registrationFlowResponse != null
                     && params.userPasswordAuth != null
                     /* Avoid infinite loop */
                     && params.userPasswordAuth.session.isNullOrEmpty()
             ) {
-                try {
-                    MoshiProvider.providesMoshi()
-                            .adapter(RegistrationFlowResponse::class.java)
-                            .fromJson(throwable.errorBody)
-                } catch (e: Exception) {
-                    null
-                }?.let {
-                    // Retry with authentication
-                    try {
-                        val req = executeRequest<KeysQueryResponse>(eventBus) {
-                            apiCall = cryptoApi.uploadSigningKeys(
-                                    uploadQuery.copy(auth = params.userPasswordAuth.copy(session = it.session))
-                            )
-                        }
-                        if (req.failures?.isNotEmpty() == true) {
-                            throw UploadSigningKeys(req.failures)
-                        }
-                        return
-                    } catch (failure: Throwable) {
-                        throw failure
-                    }
+                // Retry with authentication
+                val req = executeRequest<KeysQueryResponse>(eventBus) {
+                    apiCall = cryptoApi.uploadSigningKeys(
+                            uploadQuery.copy(auth = params.userPasswordAuth.copy(session = registrationFlowResponse.session))
+                    )
                 }
+                if (req.failures?.isNotEmpty() == true) {
+                    throw UploadSigningKeys(req.failures)
+                }
+            } else {
+                // Other error
+                throw throwable
             }
-            // Other error
-            throw throwable
         }
     }
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/verification/DefaultOutgoingSASDefaultVerificationTransaction.kt

@@ -138,7 +138,7 @@ internal class DefaultOutgoingSASDefaultVerificationTransaction(
 
     override fun onVerificationAccept(accept: ValidVerificationInfoAccept) {
         Timber.v("## SAS O: onVerificationAccept id:$transactionId")
-        if (state != VerificationTxState.Started) {
+        if (state != VerificationTxState.Started && state !=  VerificationTxState.SendingStart) {
             Timber.e("## SAS O: received accept request from invalid state $state")
             cancel(CancelCode.UnexpectedMessage)
             return
@@ -148,7 +148,7 @@ internal class DefaultOutgoingSASDefaultVerificationTransaction(
                 || !KNOWN_HASHES.contains(accept.hash)
                 || !KNOWN_MACS.contains(accept.messageAuthenticationCode)
                 || accept.shortAuthenticationStrings.intersect(KNOWN_SHORT_CODES).isEmpty()) {
-            Timber.e("## SAS O: received accept request from invalid state")
+            Timber.e("## SAS O: received invalid accept")
             cancel(CancelCode.UnknownMethod)
             return
         }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/crypto/verification/VerificationTransportToDevice.kt

@@ -117,6 +117,7 @@ internal class VerificationTransportToDevice(
                                  onDone: (() -> Unit)?) {
         Timber.d("## SAS sending msg type $type")
         Timber.v("## SAS sending msg info $verificationInfo")
+        val stateBeforeCall = tx?.state
         val tx = tx ?: return
         val contentMap = MXUsersDevicesMap<Any>()
         val toSendToDeviceObject = verificationInfo.toSendToDeviceObject()
@@ -132,7 +133,11 @@ internal class VerificationTransportToDevice(
                             if (onDone != null) {
                                 onDone()
                             } else {
-                                tx.state = nextState
+                                // we may have received next state (e.g received accept in sending_start)
+                                // We only put next state if the state was what is was before we started
+                                if (tx.state == stateBeforeCall) {
+                                    tx.state = nextState
+                                }
                             }
                         }
 

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/database/mapper/RoomSummaryMapper.kt

@@ -53,6 +53,7 @@ internal class RoomSummaryMapper @Inject constructor(private val timelineEventMa
                 canonicalAlias = roomSummaryEntity.canonicalAlias,
                 aliases = roomSummaryEntity.aliases.toList(),
                 isEncrypted = roomSummaryEntity.isEncrypted,
+                encryptionEventTs = roomSummaryEntity.encryptionEventTs,
                 typingRoomMemberIds = roomSummaryEntity.typingUserIds.toList(),
                 breadcrumbsIndex = roomSummaryEntity.breadcrumbsIndex,
                 roomEncryptionTrustLevel = roomSummaryEntity.roomEncryptionTrustLevel,

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/database/model/RoomSummaryEntity.kt

@@ -48,6 +48,7 @@ internal open class RoomSummaryEntity(
         // this is required for querying
         var flatAliases: String = "",
         var isEncrypted: Boolean = false,
+        var encryptionEventTs: Long? = 0,
         var typingUserIds: RealmList<String> = RealmList(),
         var roomEncryptionTrustLevelStr: String? = null,
         var inviterId: String? = null

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/di/NetworkModule.kt

@@ -57,7 +57,7 @@ internal object NetworkModule {
     @Provides
     @JvmStatic
     fun providesCurlLoggingInterceptor(): CurlLoggingInterceptor {
-        return CurlLoggingInterceptor(HttpLoggingInterceptor.Logger.DEFAULT)
+        return CurlLoggingInterceptor()
     }
 
     @MatrixScope

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/eventbus/EventBusTimberLogger.kt

@@ -0,0 +1,31 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package im.vector.matrix.android.internal.eventbus
+
+import org.greenrobot.eventbus.Logger
+import timber.log.Timber
+import java.util.logging.Level
+
+class EventBusTimberLogger : Logger {
+    override fun log(level: Level, msg: String) {
+        Timber.d(msg)
+    }
+
+    override fun log(level: Level, msg: String, th: Throwable) {
+        Timber.e(th, msg)
+    }
+}

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/identity/IdentityPingApi.kt

@@ -0,0 +1,34 @@
+/*
+ * Copyright (c) 2020 New Vector Ltd
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package im.vector.matrix.android.internal.identity
+
+import im.vector.matrix.android.internal.network.NetworkConstants
+import retrofit2.Call
+import retrofit2.http.GET
+
+internal interface IdentityPingApi {
+
+    /**
+     * https://matrix.org/docs/spec/client_server/r0.4.0.html#server-discovery
+     * Simple ping call to check if server alive
+     *
+     * Ref: https://matrix.org/docs/spec/identity_service/unstable#status-check
+     *
+     * @return 200 in case of success
+     */
+    @GET(NetworkConstants.URI_API_PREFIX_IDENTITY)
+    fun ping(): Call<Unit>
+}

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/network/NetworkConstants.kt

@@ -26,4 +26,10 @@ internal object NetworkConstants {
     // Media
     private const val URI_API_MEDIA_PREFIX_PATH = "_matrix/media"
     const val URI_API_MEDIA_PREFIX_PATH_R0 = "$URI_API_MEDIA_PREFIX_PATH/r0/"
+
+    // Identity server
+    const val URI_IDENTITY_PATH = "_matrix/identity/api/v1/"
+    const val URI_IDENTITY_PATH_V2 = "_matrix/identity/v2/"
+
+    const val URI_API_PREFIX_IDENTITY = "_matrix/identity/api/v1"
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/network/Request.kt

@@ -17,6 +17,7 @@
 package im.vector.matrix.android.internal.network
 
 import im.vector.matrix.android.api.failure.Failure
+import im.vector.matrix.android.api.failure.shouldBeRetried
 import kotlinx.coroutines.CancellationException
 import kotlinx.coroutines.delay
 import org.greenrobot.eventbus.EventBus
@@ -46,7 +47,7 @@ internal class Request<DATA>(private val eventBus: EventBus?) {
                 throw response.toFailure(eventBus)
             }
         } catch (exception: Throwable) {
-            if (isRetryable && currentRetryCount++ < maxRetryCount && exception is IOException) {
+            if (isRetryable && currentRetryCount++ < maxRetryCount && exception.shouldBeRetried()) {
                 delay(currentDelay)
                 currentDelay = (currentDelay * 2L).coerceAtMost(maxDelay)
                 return execute()

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/network/UserAgentHolder.kt

@@ -51,7 +51,7 @@ internal class UserAgentHolder @Inject constructor(private val context: Context,
             appName = pm.getApplicationLabel(appInfo).toString()
 
             val pkgInfo = pm.getPackageInfo(context.applicationContext.packageName, 0)
-            appVersion = pkgInfo.versionName
+            appVersion = pkgInfo.versionName ?: ""
 
             // Use appPackageName instead of appName if appName contains any non-ASCII character
             if (!appName.matches("\\A\\p{ASCII}*\\z".toRegex())) {

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/session/SessionComponent.kt

@@ -36,6 +36,7 @@ import im.vector.matrix.android.internal.session.filter.FilterModule
 import im.vector.matrix.android.internal.session.group.GetGroupDataWorker
 import im.vector.matrix.android.internal.session.group.GroupModule
 import im.vector.matrix.android.internal.session.homeserver.HomeServerCapabilitiesModule
+import im.vector.matrix.android.internal.session.openid.OpenIdModule
 import im.vector.matrix.android.internal.session.profile.ProfileModule
 import im.vector.matrix.android.internal.session.pushers.AddHttpPusherWorker
 import im.vector.matrix.android.internal.session.pushers.PushersModule
@@ -70,6 +71,7 @@ import im.vector.matrix.android.internal.util.MatrixCoroutineDispatchers
             CacheModule::class,
             CryptoModule::class,
             PushersModule::class,
+            OpenIdModule::class,
             AccountDataModule::class,
             ProfileModule::class,
             SessionAssistedInjectModule::class,

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/session/SessionModule.kt

@@ -49,6 +49,7 @@ import im.vector.matrix.android.internal.di.SessionId
 import im.vector.matrix.android.internal.di.Unauthenticated
 import im.vector.matrix.android.internal.di.UserId
 import im.vector.matrix.android.internal.di.UserMd5
+import im.vector.matrix.android.internal.eventbus.EventBusTimberLogger
 import im.vector.matrix.android.internal.network.AccessTokenInterceptor
 import im.vector.matrix.android.internal.network.DefaultNetworkConnectivityChecker
 import im.vector.matrix.android.internal.network.FallbackNetworkCallbackStrategy
@@ -205,7 +206,10 @@ internal abstract class SessionModule {
         @Provides
         @SessionScope
         fun providesEventBus(): EventBus {
-            return EventBus.builder().build()
+            return EventBus
+                    .builder()
+                    .logger(EventBusTimberLogger())
+                    .build()
         }
 
         @JvmStatic

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/session/account/AccountAPI.kt

@@ -16,7 +16,6 @@
 
 package im.vector.matrix.android.internal.session.account
 
-import im.vector.matrix.android.api.session.account.model.ChangePasswordParams
 import im.vector.matrix.android.internal.network.NetworkConstants
 import retrofit2.Call
 import retrofit2.http.Body
@@ -30,4 +29,12 @@ internal interface AccountAPI {
      */
     @POST(NetworkConstants.URI_API_PREFIX_PATH_R0 + "account/password")
     fun changePassword(@Body params: ChangePasswordParams): Call<Unit>
+
+    /**
+     * Deactivate the user account
+     *
+     * @param params the deactivate account params
+     */
+    @POST(NetworkConstants.URI_API_PREFIX_PATH_R0 + "account/deactivate")
+    fun deactivate(@Body params: DeactivateAccountParams): Call<Unit>
 }

matrix-sdk-android/src/main/java/im/vector/matrix/android/internal/session/account/AccountModule.kt

@@ -39,6 +39,9 @@ internal abstract class AccountModule {
     @Binds
     abstract fun bindChangePasswordTask(task: DefaultChangePasswordTask): ChangePasswordTask
 
+    @Binds
+    abstract fun bindDeactivateAccountTask(task: DefaultDeactivateAccountTask): DeactivateAccountTask
+
     @Binds
     abstract fun bindAccountService(service: DefaultAccountService): AccountService
 }