kodjodevf-mangayomi-extensions/CONTRIBUTING.md

# Contributing

This guide have some instructions and tips on how to create a new Mangayomi extension.

## Prerequisites

Before you start, please note that the ability to use following technologies is **required**.

- [Flutter development](https://flutter.dev/)
- [Dart](https://dart.dev/)
- Web scraping
    - [HTML](https://developer.mozilla.org/en-US/docs/Web/HTML)
    - [CSS selectors](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors)
    - [Xpath](https://www.blog.datahut.co/post/xpath-for-web-scraping-step-by-step-tutorial)

This project uses the capabilities of the [dart_eval](https://pub.dev/packages/dart_eval) package

### Tools
- [Android Studio](https://developer.android.com/studio)
- [VS Code](https://code.visualstudio.com/)
- Any platform emulator

### Writing your extension
1. Clone [Mangayomi](https://github.com/kodjodevf/mangayomi) next, read the README file to learn how to run the project.
2. Open the project then go to : 
```console
lib
└── sources
      └── source_test.dart
```
and you will see this : 

```bash
import 'package:mangayomi/models/source.dart';

//For testing purposes, set to true
const useTestSourceCode = false;

final testSourceModelList = [
  Source(
      id: "Test Source".hashCode,
      name: "Test Source",
      // Example: https://gogoanime3.net
      baseUrl: "",
      // source code
      sourceCode: testSourceCode,
      // Example: en
      lang: "",
      // Example: false for true for manga
      isManga: false)
];

const testSourceCode = r'''
import 'package:mangayomi/bridge_lib.dart';
import 'dart:convert';

class TestSource extends MProvider {
  TestSource();

  @override
  Future<MPages> getPopular(MSource source, int page) async {
    // TODO: implement
  }

  @override
  Future<MPages> getLatestUpdates(MSource source, int page) async {
    // TODO: implement
  }

  @override
  Future<MPages> search(
      MSource source, String query, int page, FilterList filterList) async {
    // TODO: implement
  }

  @override
  Future<MManga> getDetail(MSource source, String url) async {
    // TODO: implement
  }
  
  // For anime episode video list
  @override
  Future<List<MVideo>> getVideoList(MSource source, String url) async {
    // TODO: implement
  }

  // For manga chapter pages
  @override
  Future<List<String>> getPageList(MSource source, String url) {
    // TODO: implement
  }

  @override
  List<dynamic> getFilterList(MSource source) {
    // TODO: implement
  }

  @override
  List<dynamic> getSourcePreferences(MSource source) {
    // TODO: implement
  }
}

TestSource main() {
  return TestSource();
}

''';

```
For testing your source set `useTestSourceCode` to true.
During the development it is recommended (but not necessary) to write it directly mangayomi application project.
Once extension is ready you can relocate your code into `mangayomi-extension` project in a `src` or `multisrc` package and create a Pull Request.

### Source 

| Field | Description |
| ----- | ----------- |
| `name` | Name displayed in the "Sources" tab in Mangayomi. |
| `baseUrl` | Base URL of the source without any trailing slashes. |
| `apiUrl` | (Optional, defaults is empty) Api URL of the source with trailing slashes. |
| `lang` | An ISO 639-1 compliant language code (two letters in lower case in most cases, but can also include the country/dialect part by using a simple dash character). |
| `id` | Identifier of your source, automatically set in `Source`. It should only be manually overriden if you need to copy an existing autogenerated ID. |
| `sourceCodeUrl` | contains the URL where the extension source code can be downloaded |
| `sourceCode` | contains the extension source code |
| `isManga` | (Optional, defaults to `true`) specify source type (false for anime and true for manga)|
| `dateFormat` | (Optional, defaults is empty) |
| `iconUrl` | The extension icon URL |
| `version` | The extension version code. This must be incremented with any change to the code. |
| `dateFormatLocale` | (Optional, defaults is empty) |
| `isNsfw` | (Optional, defaults to `false`) Flag to indicate that a source contains NSFW content. |

### Extension call flow

#### Popular manga

a.k.a. the Browse source entry point in the app (invoked by tapping on the source name).

- The app calls `getPopular` which should return a `MPages` containing the first batch of found `MManga` entries.
    - This method supports pagination. When user scrolls the manga  list and more results must be fetched, the app calls it again with increasing `page` values(starting with `page=1`). This continues while `MPages.hasNextPage` is passed as `true` and `MPages.list` is not empty.
- To show the list properly, the app needs `url`, `title` and `imageUrl`. You **must** set them here. The rest of the fields could be filled later.(refer to Manga Details below).

#### Latest manga

a.k.a. the Latest source entry point in the app (invoked by tapping on the "Latest" button beside the source name).

- Similar to popular manga, but should be fetching the latest entries from a source.

#### Search manga

- When the user searches inside the app, `search` will be called and the rest of the flow is similar to what happens with `getPopular`.
    - If search functionality is not available, return `MPages([], false)`
- `getFilterList` will be called to get all filters and filter types.

### Filters

The search flow have support to filters that can be added to a `FilterList` inside the `getFilterList` method. When the user changes the filter's state, they will be passed to the `search` method, and they can be iterated to create the request (by getting the `filter.state` value, where the type varies depending on the `Filter` used). You can check the filter types available [here](https://github.com/kodjodevf/mangayomi/blob/main/lib/eval/model/filter.dart) and in the table below.

| Filter | Description |
| ------ | ----------- |
| `HeaderFilter` | A simple header. Useful for separating sections in the list or showing any note or warning to the user. |
| `SeparatorFilter` | A line separator. Useful for visual distinction between sections. |
| `SelectFilter` | A select control, similar to HTML's `<select>`. Only one item can be selected, and the state is the index of the selected one. |
| `TextFilter` | A text control, similar to HTML's `<input type="text">`. |
| `CheckBoxFilter` | A checkbox control, similar to HTML's `<input type="checkbox">`. The state is `true` if it's checked. |
| `TriStateFilter` | A enhanced checkbox control that supports an excluding state |
| `GroupFilter` | A group of filters (preferentially of the same type). The state will be a `List` with all the states. |
| `SortFilter` | A control for sorting, with support for the ordering. The state indicates which item index is selected and if the sorting is `ascending`. |

All control filters can have a default state set. It's usually recommended if the source have filters to make the initial state match the popular manga list, so when the user open the filter sheet, the state is equal and represents the current manga showing.

#### Manga Details

- When user taps on an manga, `getDetail` will be called and the results will be cached.
    - A `MManga` entry is identified by its `url`.
- `getDetail` is called to update an manga's details from when it was initialized earlier.
    - `MManga.title` is a string containing title.
    - `MManga.description` is a string containing description.
    - `MManga.author` is a string containing author.
    - `MManga.genre` contain list of all genres.
    - `MManga.status` is an "enum" value.
        - To get the status in enum value from a status string, you can use a `parseStatus` function like in the example below.
        `Status` parseStatus(`String status`, `List<dynamic> statusList`)
        ```bash
        final statusList = [
              { "ongoing": 0,
                "complete": 1,
                "hiatus": 2,
                "canceled": 3,
                "publishingFinished": 4,
              }
            ];
        final status = parseStatus('ongoing', statusList);
        print(status); // Status.ongoing
        ```
      Refer to [the values in the `MManga` model](https://github.com/kodjodevf/mangayomi/blob/main/lib/models/manga.dart).
    - During a backup, only `url` and `title` are stored. To restore the rest of the manga data, the app calls `getDetail`, so all fields should be (re)filled in if possible.
    - If a `MManga` is cached `getDetail` will be only called when the user does a manual update(Swipe-to-Refresh).
    - `MManga.chapters` contain list of all manga chapters.
    - `MChapter.name` is a string containing a chapter name.
    - `MChapter.url` is a string containing a chapter url.
    - `MChapter.scanlator` is a string containing a chapter scanlator.
    - `MChapter.dateUpload` is a string containing date **expressed in millisecondsSinceEpoch**.
        - To get the time in millisecondsSinceEpoch from a date string, you can use a `parseDates` function like in the example below.
        `List<dynamic>` parseDates(`List<dynamic> values`, `String dateFormat`, `String dateFormatLocale`,)
          ```bash
          final dates = parseDates(["2023-12-10T11:49:02+000"], "yyyy-MM-dd'T'HH:mm:ss+SSS", "en_US");
          ```
        - If you don't pass `MChapter.dateUpload` and leave it null, the app will use the default date instead, but it's recommended to always fill it if it's available.


#### Chapter pages

- When user opens an chapter, `getPageList` will be called and it will return a `List<String>` that are used by the reader.

#### Episode Videos

- When user opens an episode, `getVideoList` will be called and it will return a `List<MVideo>` that are used by the player.

## Example sources that can help you understand how to create your source

- [Example](https://github.com/kodjodevf/mangayomi-extensions/tree/main/anime/src/en/kisskh)
  of Json API usage.
- [Example](https://github.com/kodjodevf/mangayomi-extensions/tree/main/manga/src/en/mangahere)
  of pure HTML parsing using xpath selector.

  
## Some functions already available and usable


### http  

Return a body response as `String`
```bash
- Simple request

String res = await http('GET', json.encode({'url': 'http://example.com'}));

- With headers 

String res = await http('GET', json.encode({'url': 'http://example.com','headers':{'Referer': 'http://example.com'}}));

- With body

String res = await http('POST', json.encode({'url': 'http://example.com','body':{'name':'test'}}));

- With FormBuilder

String res = await http('POST',json.encode({'useFormBuilder': true,'body': {'name':'test'},'url': 'http://example.com'}));
```

### xpath selector
Return result as `List<String>`

Example: 
```bash
final String htmlString = '''
<html lang="en">
<body>
<div><a href='https://github.com/kodjodevf'>author</a></div>
<div class="head">div head</div>
<div class="container">
    <table>
        <tbody>
          <tr>
              <td id="td1" class="first1">1</td>
              <td id="td2" class="first1">2</td>
              <td id="td3" class="first2">3</td>
              <td id="td4" class="first2 form">4</td>
              <td id="td5" class="second1">one</td>
              <td id="td6" class="second1">two</td>
              <td id="td7" class="second2">three</td>
              <td id="td8" class="second2">four</td>
          </tr>
        </tbody>
    </table>
</div>
<div class="end">end</div>
</body>
</html>
''';


  List<String> xpathRes = xpath(htmlString,'//div/a/@href');
  print(xpathRes); // [https://github.com/kodjodevf]
  print(xpathRes.first); // https://github.com/kodjodevf

```
### DOM selector

Example: 
```bash
final String htmlString = '''
<html lang="en">
<body>
<div><a href='https://github.com/kodjodevf'>author</a></div>
<div class="head">div head</div>
<div class="container">
    <table>
        <tbody>
          <tr>
              <td id="td1" class="first1">1</td>
              <td id="td2" class="first1">2</td>
              <td id="td3" class="first2">3</td>
              <td id="td4" class="first2 form">4</td>
              <td id="td5" class="second1">one</td>
              <td id="td6" class="second1">two</td>
              <td id="td7" class="second2">three</td>
              <td id="td8" class="second2">four</td>
          </tr>
        </tbody>
    </table>
</div>
<div class="end">end</div>
</body>
</html>
''';


  MDocument document = parseHtml(htmlString);
  print(document.selecFirst("a").attr("href")); // https://github.com/kodjodevf
  print(document.selecFirst("td").text); // 1

  See [`MDocument` model](https://github.com/kodjodevf/mangayomi/blob/main/lib/eval/model/document.dart) and  [`MElement` model](https://github.com/kodjodevf/mangayomi/blob/main/lib/eval/model/element.dart) to see aivable methods.

```


### String utils
- `String` substringAfter(`String text`, `String pattern`)
- `String` substringAfterLast(`String text`, `String pattern`)
- `String` substringBefore(`String text`, `String pattern`)
- `String` substringBeforeLast(`String text`, `String pattern`)

### Crypto utils
- `String` evalJs(`String code`);
- `String` deobfuscateJsPassword(`String inputString`)
- `String` encryptAESCryptoJS(`String plainText`, `String passphrase`)
- `String` decryptAESCryptoJS(`String encrypted`, `String passphrase`)
- `String` cryptoHandler(`String text`, `String iv`, `String secretKeyString`, `bool encrypt`)

## Help

If you need a help or have some questions, ask a community in our [Discord server](https://discord.com/invite/EjfBuYahsP).