Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All: Add new encryption methods based on native crypto libraries #10696

Merged
merged 78 commits into from
Oct 26, 2024
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
71f0204
Add basic framework for native encryption
wh201906 Jul 7, 2024
17aa0fd
Remove class
wh201906 Jul 8, 2024
a2be9f1
Rename to Crypto
wh201906 Jul 8, 2024
17badd5
Rename to Krypto
wh201906 Jul 11, 2024
80a764b
Merge branch 'dev' into wh201906/native_encryption
wh201906 Jul 19, 2024
4c8c2f3
Rename
wh201906 Jul 19, 2024
8387c96
Remove type any
wh201906 Jul 19, 2024
0276041
Add common interface for the Buffer in Node.js and react-native-buffer
wh201906 Jul 19, 2024
832e589
Clean up
wh201906 Jul 20, 2024
373ecac
Fix the type of digest
wh201906 Jul 21, 2024
d3bd657
Clean up
wh201906 Jul 21, 2024
e4eb58c
Rename
wh201906 Jul 23, 2024
f2ed473
Use enum for Digest
wh201906 Jul 23, 2024
0182cce
Merge branch 'dev' into wh201906/native_encryption
wh201906 Jul 26, 2024
48b54ec
Add a linter ignore rule for crypto terms
wh201906 Jul 28, 2024
70db354
Add encryption core and string wrapper
wh201906 Jul 28, 2024
52f8244
Clean up
wh201906 Aug 3, 2024
7b7595f
Integrate into EncryptionService
wh201906 Aug 3, 2024
26faf26
Add tests
wh201906 Aug 3, 2024
63f6ce9
Enable native encryption
wh201906 Aug 3, 2024
295ef6f
Merge branch 'dev' into wh201906/native_encryption
wh201906 Aug 3, 2024
9d39eb7
Tuning parameters
wh201906 Aug 4, 2024
bfc3ecb
Add performance test for crypto
wh201906 Aug 4, 2024
73b4c5a
Resolve some review suggestions
wh201906 Aug 6, 2024
bf0f126
Use featureFlag for enabling the new encryption method
wh201906 Aug 10, 2024
61dd12d
Set encryption chunk size to method-specific
wh201906 Aug 10, 2024
1536a04
Merge branch 'dev' to resolve conflicts
wh201906 Aug 10, 2024
8b20d12
Merge branch 'dev' into wh201906/native_encryption
laurent22 Aug 10, 2024
04dfc8b
Empty commit for triggering CI
wh201906 Aug 11, 2024
b912dca
Move featureFlag.useBetaEncryptionMethod to sync section
wh201906 Aug 11, 2024
2adf438
Remove translations in featureFlag item
wh201906 Aug 12, 2024
95281e7
Fix tests
wh201906 Aug 13, 2024
d780b09
Remove internal methods from interface definition
wh201906 Aug 20, 2024
cbf2563
Remove null value in parameter "salt"
wh201906 Aug 20, 2024
d7a3b9c
Remove null value in parameter "iv"
wh201906 Aug 20, 2024
885a4b6
Remove null value in parameter "associatedData"
wh201906 Aug 20, 2024
b96bd0d
Move default parameters in crypto.ts to encrypt()/decrypt() implement…
wh201906 Aug 20, 2024
b99f9fc
Modify EncryptionOptions
wh201906 Aug 20, 2024
7026dc2
Move associatedData from encrypt()/decrypt() to EncryptionParameters
wh201906 Aug 20, 2024
bca6e4d
Clean up
wh201906 Aug 20, 2024
d9d8eae
Rename a method parameter
wh201906 Aug 20, 2024
bb69f50
Move featureFlag.useBetaEncryptionMethod to sync.advanced
wh201906 Aug 20, 2024
bc9605e
Better chunkSize
wh201906 Aug 20, 2024
91f9c9b
Add type for encryption handler
wh201906 Aug 20, 2024
b329c0c
Add test cases for invalid ciphertext
wh201906 Aug 20, 2024
d0140da
Migrate to web crypto from Node crypto
personalizedrefrigerator Aug 21, 2024
41e7918
Enable integrated tests on web
personalizedrefrigerator Aug 21, 2024
6087f77
Revert an unintended change
wh201906 Aug 21, 2024
72366e2
Merge branch 'dev' into wh201906/native_encryption
wh201906 Aug 22, 2024
dfefff7
Fix randomBytes for a large output size
personalizedrefrigerator Aug 22, 2024
2fc951d
Merge branch 'personalizedrefrigerator:downstream-pr/use-web-crypto' …
wh201906 Aug 22, 2024
1f9525d
Remove digest name mapping
wh201906 Aug 22, 2024
10c185f
Less calculation and branches in loop
wh201906 Aug 22, 2024
d56dfc0
Remove parameter validation in hot path
wh201906 Aug 22, 2024
ab22f04
Reduce overhead
wh201906 Aug 22, 2024
701c313
Clean up .eslintrc.js
wh201906 Aug 22, 2024
b637071
Enhance nonce generation
wh201906 Aug 23, 2024
58aec1d
Merge branch 'dev' into wh201906/native_encryption
wh201906 Aug 23, 2024
cb9b077
Add digest name mapping for QuickCrypto.createHash()
wh201906 Aug 24, 2024
1a2c3d2
Reduce chunk size to prevent potential freezing
wh201906 Aug 24, 2024
2855ec7
Increase nonce size before hashing
wh201906 Aug 24, 2024
7dc4eec
Replace console with logger in crypto integration tests
wh201906 Aug 28, 2024
9a7b310
Move duplicate code to cryptoShared.ts
wh201906 Aug 28, 2024
2b42bbf
Fix timestamp bytes in nonce
wh201906 Aug 28, 2024
8cfd6dd
Merge branch 'dev' into wh201906/native_encryption
wh201906 Aug 28, 2024
6de1015
Merge branch 'dev' into wh201906/native_encryption
laurent22 Aug 31, 2024
bf0d512
Clean up
wh201906 Sep 6, 2024
8def1be
Merge branch 'dev' to resolve conflicts
wh201906 Sep 11, 2024
115b502
Fix yarn.lock
wh201906 Sep 11, 2024
8fcc8ee
Use it.each to make test cases independent of each other
wh201906 Sep 12, 2024
edd2a90
Add test cases for new encryption methods in Synchronizer.e2ee.test.ts
wh201906 Sep 12, 2024
a786478
Merge branch 'dev' to resolve conflicts
wh201906 Sep 16, 2024
d8f243a
Merge branch 'dev' to resolve conflicts
wh201906 Sep 18, 2024
e78cc10
Merge branch 'dev' to resolve conflicts
wh201906 Sep 20, 2024
5934ccb
Merge branch 'dev' to resolve conflicts
wh201906 Sep 21, 2024
9e22379
Merge branch 'dev' to resolve conflicts
wh201906 Sep 22, 2024
002f7de
Merge branch 'dev' to resolve conflicts
wh201906 Oct 15, 2024
98acc2a
Merge branch 'dev' to resolve conflicts
wh201906 Oct 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
Expand Up @@ -688,6 +688,7 @@ packages/app-mobile/root.js
packages/app-mobile/services/AlarmServiceDriver.android.js
packages/app-mobile/services/AlarmServiceDriver.ios.js
packages/app-mobile/services/e2ee/RSA.react-native.js
packages/app-mobile/services/e2ee/crypto.js
packages/app-mobile/services/plugins/PlatformImplementation.js
packages/app-mobile/services/profiles/index.js
packages/app-mobile/services/voiceTyping/vosk.android.js
Expand Down Expand Up @@ -1028,6 +1029,7 @@ packages/lib/services/debug/populateDatabase.js
packages/lib/services/e2ee/EncryptionService.test.js
packages/lib/services/e2ee/EncryptionService.js
packages/lib/services/e2ee/RSA.node.js
packages/lib/services/e2ee/crypto.js
packages/lib/services/e2ee/ppk.test.js
packages/lib/services/e2ee/ppk.js
packages/lib/services/e2ee/ppkTestUtils.js
Expand Down
4 changes: 3 additions & 1 deletion .eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ module.exports = {
'tinymce': 'readonly',

'JSX': 'readonly',

'BufferEncoding': 'readonly',
},
'parserOptions': {
'ecmaVersion': 2018,
Expand Down Expand Up @@ -270,7 +272,7 @@ module.exports = {
selector: 'enumMember',
format: null,
'filter': {
'regex': '^(GET|POST|PUT|DELETE|PATCH|HEAD|SQLite|PostgreSQL|ASC|DESC|E2EE|OR|AND|UNION|INTERSECT|EXCLUSION|INCLUSION|EUR|GBP|USD|SJCL.*|iOS)$',
'regex': '^(GET|POST|PUT|DELETE|PATCH|HEAD|SQLite|PostgreSQL|ASC|DESC|E2EE|OR|AND|UNION|INTERSECT|EXCLUSION|INCLUSION|EUR|GBP|USD|SJCL.*|iOS|sha1|sha224|sha256|sha384|sha512|ripemd160)$',
'match': true,
},
},
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -667,6 +667,7 @@ packages/app-mobile/root.js
packages/app-mobile/services/AlarmServiceDriver.android.js
packages/app-mobile/services/AlarmServiceDriver.ios.js
packages/app-mobile/services/e2ee/RSA.react-native.js
packages/app-mobile/services/e2ee/crypto.js
packages/app-mobile/services/plugins/PlatformImplementation.js
packages/app-mobile/services/profiles/index.js
packages/app-mobile/services/voiceTyping/vosk.android.js
Expand Down Expand Up @@ -1007,6 +1008,7 @@ packages/lib/services/debug/populateDatabase.js
packages/lib/services/e2ee/EncryptionService.test.js
packages/lib/services/e2ee/EncryptionService.js
packages/lib/services/e2ee/RSA.node.js
packages/lib/services/e2ee/crypto.js
packages/lib/services/e2ee/ppk.test.js
packages/lib/services/e2ee/ppk.js
packages/lib/services/e2ee/ppkTestUtils.js
Expand Down
1 change: 1 addition & 0 deletions packages/app-mobile/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@
"react-native-paper": "5.12.3",
"react-native-popup-menu": "0.16.1",
"react-native-quick-actions": "0.3.13",
"react-native-quick-crypto": "0.7.1",
"react-native-rsa-native": "2.0.5",
"react-native-safe-area-context": "4.10.1",
"react-native-securerandom": "1.0.1",
Expand Down
52 changes: 52 additions & 0 deletions packages/app-mobile/services/e2ee/crypto.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
import { Crypto, CryptoBuffer, Digest } from '@joplin/lib/services/e2ee/types';
import QuickCrypto from 'react-native-quick-crypto';
import { HashAlgorithm } from 'react-native-quick-crypto/lib/typescript/keys';

type DigestNameMap = Record<Digest, HashAlgorithm>;

const crypto: Crypto = {

getCiphers: () => {
return QuickCrypto.getCiphers();
},

getHashes: () => {
return QuickCrypto.getHashes();
},

randomBytes: async (size: number) => {
return new Promise((resolve, reject) => {
QuickCrypto.randomBytes(size, (error, result) => {
if (error) {
reject(error);
} else {
resolve(result);
}
});
});
},

pbkdf2Raw: async (password: string, salt: CryptoBuffer, iterations: number, keylen: number, digest: Digest) => {
const digestNameMap: DigestNameMap = {
sha1: 'SHA-1',
sha224: 'SHA-224',
sha256: 'SHA-256',
sha384: 'SHA-384',
sha512: 'SHA-512',
ripemd160: 'RIPEMD-160',
};
const rnqcDigestName = digestNameMap[digest];

return new Promise((resolve, reject) => {
QuickCrypto.pbkdf2(password, salt, iterations, keylen, rnqcDigestName, (error, result) => {
if (error) {
reject(error);
} else {
resolve(result);
}
});
});
},
};

export default crypto;
2 changes: 2 additions & 0 deletions packages/app-mobile/utils/shim-init-react.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,12 @@ import { getLocales } from 'react-native-localize';
import { setLocale, defaultLocale, closestSupportedLocale } from '@joplin/lib/locale';
import type SettingType from '@joplin/lib/models/Setting';
import injectedJs from './injectedJs';
import crypto from '../services/e2ee/crypto';

export default function shimInit() {
shim.Geolocation = GeolocationReact;
shim.sjclModule = require('@joplin/lib/vendor/sjcl-rn.js');
shim.crypto = crypto;

shim.fsDriver = () => {
if (!shim.fsDriver_) {
Expand Down
30 changes: 30 additions & 0 deletions packages/lib/services/e2ee/crypto.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
import { Crypto, Digest } from './types';
import { promisify } from 'util';
import {
getCiphers as nodeGetCiphers, getHashes as nodeGetHashes,
randomBytes as nodeRandomBytes,
pbkdf2 as nodePbkdf2,
} from 'crypto';

const crypto: Crypto = {

getCiphers: () => {
return nodeGetCiphers();
},

getHashes: () => {
return nodeGetHashes();
},

randomBytes: async (size: number) => {
const randomBytesAsync = promisify(nodeRandomBytes);
return randomBytesAsync(size);
},

pbkdf2Raw: async (password: string, salt: Buffer, iterations: number, keylen: number, digest: Digest) => {
const pbkdf2Async = promisify(nodePbkdf2);
return pbkdf2Async(password, salt, iterations, keylen, digest);
},
};

export default crypto;
21 changes: 21 additions & 0 deletions packages/lib/services/e2ee/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,24 @@ export interface RSA {
publicKey(rsaKeyPair: RSAKeyPair): string;
privateKey(rsaKeyPair: RSAKeyPair): string;
}

export interface Crypto {
getCiphers(): string[];
getHashes(): string[];
randomBytes(size: number): Promise<CryptoBuffer>;
pbkdf2Raw(password: string, salt: CryptoBuffer, iterations: number, keylen: number, digest: Digest): Promise<CryptoBuffer>;
}

export interface CryptoBuffer extends Uint8Array {
toString(encoding?: BufferEncoding, start?: number, end?: number): string;
}

// From react-native-quick-crypto.HashAlgorithm, but use the hash name style in node:crypto
export enum Digest {
sha1 = 'sha1',
sha224 = 'sha224',
sha256 = 'sha256',
sha384 = 'sha384',
sha512 = 'sha512',
ripemd160 = 'ripemd160',
laurent22 marked this conversation as resolved.
Show resolved Hide resolved
}
2 changes: 2 additions & 0 deletions packages/lib/shim-init-node.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import { ResourceEntity } from './services/database/types';
import { TextItem } from 'pdfjs-dist/types/src/display/api';
import replaceUnsupportedCharacters from './utils/replaceUnsupportedCharacters';
import { FetchBlobOptions } from './types';
import crypto from './services/e2ee/crypto';

import FileApiDriverLocal from './file-api-driver-local';
import * as mimeUtils from './mime-utils';
Expand Down Expand Up @@ -135,6 +136,7 @@ function shimInit(options: ShimInitOptions = null) {
shim.Geolocation = GeolocationNode;
shim.FormData = require('form-data');
shim.sjclModule = require('./vendor/sjcl.js');
shim.crypto = crypto;
shim.electronBridge_ = options.electronBridge;

shim.fsDriver = () => {
Expand Down
3 changes: 3 additions & 0 deletions packages/lib/shim.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import * as React from 'react';
import { NoteEntity, ResourceEntity } from './services/database/types';
import type FsDriverBase from './fs-driver-base';
import type FileApiDriverLocal from './file-api-driver-local';
import { Crypto } from './services/e2ee/types';

export interface CreateResourceFromPathOptions {
resizeLargeImages?: 'always' | 'never' | 'ask';
Expand Down Expand Up @@ -286,6 +287,8 @@ const shim = {
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- Old code before rule was applied
sjclModule: null as any,

crypto: null as Crypto,

// eslint-disable-next-line @typescript-eslint/no-explicit-any -- Old code before rule was applied
randomBytes: async (_count: number): Promise<any> => {
throw new Error('Not implemented: randomBytes');
Expand Down
4 changes: 3 additions & 1 deletion packages/tools/cspell/dictionary4.txt
Original file line number Diff line number Diff line change
Expand Up @@ -110,4 +110,6 @@ ENOTFOUND
Scaleway
Inkscape
Ionicon
Stormlikes
Stormlikes
ripemd
rnqc
51 changes: 50 additions & 1 deletion yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -4367,6 +4367,16 @@ __metadata:
languageName: node
linkType: hard

"@craftzdog/react-native-buffer@npm:^6.0.5":
version: 6.0.5
resolution: "@craftzdog/react-native-buffer@npm:6.0.5"
dependencies:
ieee754: ^1.2.1
react-native-quick-base64: ^2.0.5
checksum: 921b8bc7f84778e355e81e475792399276d611a346a7e51b6266a45cf4aa82194beb3a8106af796ed143d958c8476070c59e3720c0eec0a3c31e368fbb08b350
languageName: node
linkType: hard

"@cronvel/get-pixels@npm:^3.4.0":
version: 3.4.0
resolution: "@cronvel/get-pixels@npm:3.4.0"
Expand Down Expand Up @@ -6902,6 +6912,7 @@ __metadata:
react-native-paper: 5.12.3
react-native-popup-menu: 0.16.1
react-native-quick-actions: 0.3.13
react-native-quick-crypto: 0.7.1
react-native-rsa-native: 2.0.5
react-native-safe-area-context: 4.10.1
react-native-securerandom: 1.0.1
Expand Down Expand Up @@ -36117,6 +36128,31 @@ __metadata:
languageName: node
linkType: hard

"react-native-quick-base64@npm:^2.0.5":
version: 2.1.2
resolution: "react-native-quick-base64@npm:2.1.2"
dependencies:
base64-js: ^1.5.1
peerDependencies:
react: "*"
react-native: "*"
checksum: 46f3b26f48b26978686b0c043336220d681e6a02af5abcf3eb4ab7b9216251d1eb2fac5c559e984d963e93f54bd9f323651daac09762196815558abbd551729b
languageName: node
linkType: hard

"react-native-quick-crypto@npm:0.7.1":
version: 0.7.1
resolution: "react-native-quick-crypto@npm:0.7.1"
dependencies:
"@craftzdog/react-native-buffer": ^6.0.5
events: ^3.3.0
readable-stream: ^4.5.2
string_decoder: ^1.3.0
util: ^0.12.5
checksum: 9a7a1fb1456410db30f062078f570bc566cac36fbc165e7d8ee8677bec09fcc96923de3cf7a0464804142af242a822bf66ea22460951399b9247e4a03fcfe059
languageName: node
linkType: hard

"react-native-rsa-native@npm:2.0.5":
version: 2.0.5
resolution: "react-native-rsa-native@npm:2.0.5"
Expand Down Expand Up @@ -36833,6 +36869,19 @@ __metadata:
languageName: node
linkType: hard

"readable-stream@npm:^4.5.2":
version: 4.5.2
resolution: "readable-stream@npm:4.5.2"
dependencies:
abort-controller: ^3.0.0
buffer: ^6.0.3
events: ^3.3.0
process: ^0.11.10
string_decoder: ^1.3.0
checksum: c4030ccff010b83e4f33289c535f7830190773e274b3fcb6e2541475070bdfd69c98001c3b0cb78763fc00c8b62f514d96c2b10a8bd35d5ce45203a25fa1d33a
languageName: node
linkType: hard

"readable-stream@npm:~2.0.0":
version: 2.0.6
resolution: "readable-stream@npm:2.0.6"
Expand Down Expand Up @@ -43293,7 +43342,7 @@ __metadata:
languageName: node
linkType: hard

"util@npm:^0.12.4":
"util@npm:^0.12.4, util@npm:^0.12.5":
version: 0.12.5
resolution: "util@npm:0.12.5"
dependencies:
Expand Down
Loading