Add support for more image uploader services (#1741)

The list of links that have been uploaded is now also stored in the json format instead of csv

CHANGELOG.md

@@ -2,8 +2,10 @@
 
 ## Unversioned
 - Major: We now support image thumbnails coming from the link resolver. This feature is off by default and can be enabled in the settings with the "Show link thumbnail" setting. This feature also requires the "Show link info when hovering" setting to be enabled (#1664)
-- Major: Added image upload functionality to i.nuuls.com. This works by dragging and dropping an image into a split, or pasting an image into the text edit field. (#1332)
+- Major: Added image upload functionality to i.nuuls.com with an ability to change upload destination. This works by dragging and dropping an image into a split, or pasting an image into the text edit field. (#1332, #1741)
 - Minor: You can now open the Twitch User Card by middle-mouse clicking a username. (#1669)
+- Minor: User Popup now also includes recent user messages (#1729)
+- Minor: BetterTTV / FrankerFaceZ emote tooltips now also have emote authors' name (#1721)
 - Minor: Emotes in the emote popup are now sorted in the same order as the tab completion (#1549)
 - Minor: Removed "Online Logs" functionality as services are shut down (#1640)
 - Bugfix: Fix preview on hover not working when Animated emotes options was disabled (#1546)

docs/ENV.md

@@ -19,21 +19,6 @@ Default value: `https://braize.pajlada.com/chatterino/twitchemotes/set/%1/`
 Arguments:  
  - `%1` = Emote set ID
 
-### CHATTERINO2_IMAGE_UPLOADER_URL
-Used to change the URL that Chatterino2 uses when trying to paste an image into chat. This can be used for hosting the uploaded images yourself.  
-Default value: `https://i.nuuls.com/upload`
-
-Arguments:
- - None
-
-Notes:
- - If you want to host the images yourself. You need [Nuuls' filehost software](https://github.com/nuuls/filehost)
- - Other image hosting software is currently not supported.
-
-### CHATTERINO2_IMAGE_UPLOADER_FORM_BODY
-Used to change the name of an image form field in a request to the URL that Chatterino2 uses when trying to paste an image into chat. This can be used when your image uploading software accepts a different form field than default value.  
-Default value: `attachment`
-
 ### CHATTERINO2_TWITCH_SERVER_HOST
 String value used to change what Twitch chat server host to connect to.  
 Default value: `irc.chat.twitch.tv`

docs/IMAGEUPLOADER.md

@@ -0,0 +1,47 @@
+## Image Uploader
+You can drag and drop images to Chatterino or paste them from clipboard to upload them to an external service.
+
+By default, images are uploaded to [i.nuuls.com](https://i.nuuls.com).  
+You can change that in `Chatterino Settings -> External Tools -> Image Uploader`.  
+
+Note to advanced users: This module sends multipart-form requests via POST method, so uploading via SFTP/FTP won't work.  
+However, popular hosts like [imgur.com](https://imgur.com) are [s-ul.eu](https://s-ul.eu) supported. Scroll down to see example cofiguration.
+
+### General settings explanation:
+
+|Row|Description|
+|-|-|
+|Request URL|Link to an API endpoint, which is requested by chatterino. Any needed URL parameters should be included here.|
+|Form field|Name of a field, which contains image data.|
+|Extra headers|Extra headers, that will be included in the request. Header name and value must be separated by colon (`:`). Multiple headers need to be separated with semicolons (`;`).<br>Example: `Authorization: supaKey ; NextHeader: value` .|
+|Image link|Schema that tells where is the link in service's response. Leave empty if server's response is just the link itself. Refer to json properties by `{property}`. Supports dot-notation, example: `{property.anotherProperty}` .|
+|Deletion link|Same as above.|
+
+<br>
+
+## Examples
+
+### i.nuuls.com
+
+Simply clear all the fields.
+
+### imgur.com
+
+|Row|Description|
+|-|-|
+|Request URL|`https://api.imgur.com/3/image`|
+|Form field|`image`|
+|Extra headers|`Authorization: Client-ID c898c0bb848ca39`|
+|Image link|`{data.link}`|
+|Deletion link|`https://imgur.com/delete/{data.deletehash}`|
+
+### s-ul.eu
+
+Replace `XXXXXXXXXXXXXXX` with your API key from s-ul.eu. It can be found on [your account's configuration page](https://s-ul.eu/account/configurations).
+|Row|Description|
+|-|-|
+|Request URL|`https://s-ul.eu/api/v1/upload?wizard=true&key=XXXXXXXXXXXXXXX`|
+|Form field|`file`|
+|Extra headers||
+|Image link|`{url}`|
+|Deletion link|`https://s-ul.eu/delete.php?file={filename}&key=XXXXXXXXXXXXXXX`|

src/common/Env.cpp

@@ -57,10 +57,6 @@ Env::Env()
     , twitchEmoteSetResolverUrl(readStringEnv(
           "CHATTERINO2_TWITCH_EMOTE_SET_RESOLVER_URL",
           "https://braize.pajlada.com/chatterino/twitchemotes/set/%1/"))
-    , imageUploaderUrl(readStringEnv("CHATTERINO2_IMAGE_UPLOADER_URL",
-                                     "https://i.nuuls.com/upload"))
-    , imageUploaderFormBody(
-          readStringEnv("CHATTERINO2_IMAGE_UPLOADER_FORM_BODY", "attachment"))
     , twitchServerHost(
           readStringEnv("CHATTERINO2_TWITCH_SERVER_HOST", "irc.chat.twitch.tv"))
     , twitchServerPort(readPortEnv("CHATTERINO2_TWITCH_SERVER_PORT", 443))

src/common/Env.hpp

@@ -14,8 +14,6 @@ public:
     const QString recentMessagesApiUrl;
     const QString linkResolverUrl;
     const QString twitchEmoteSetResolverUrl;
-    const QString imageUploaderUrl;
-    const QString imageUploaderFormBody;
     const QString twitchServerHost;
     const uint16_t twitchServerPort;
     const bool twitchServerSecure;

src/common/NetworkRequest.cpp

@@ -99,6 +99,20 @@ NetworkRequest NetworkRequest::header(const char *headerName,
     return std::move(*this);
 }
 
+NetworkRequest NetworkRequest::headerList(const QStringList &headers) &&
+{
+    for (const QString &header : headers)
+    {
+        const QStringList thisHeader = header.trimmed().split(":");
+        if (thisHeader.size() == 2)
+        {
+            this->data->request_.setRawHeader(thisHeader[0].trimmed().toUtf8(),
+                                              thisHeader[1].trimmed().toUtf8());
+        }
+    }
+    return std::move(*this);
+}
+
 NetworkRequest NetworkRequest::timeout(int ms) &&
 {
     this->data->hasTimeout_ = true;

src/common/NetworkRequest.hpp

@@ -3,6 +3,7 @@
 #include "common/NetworkCommon.hpp"
 #include "common/NetworkResult.hpp"
 
+#include <QHttpMultiPart>
 #include <memory>
 
 namespace chatterino {
@@ -52,6 +53,7 @@ public:
     NetworkRequest header(const char *headerName, const char *value) &&;
     NetworkRequest header(const char *headerName, const QByteArray &value) &&;
     NetworkRequest header(const char *headerName, const QString &value) &&;
+    NetworkRequest headerList(const QStringList &headers) &&;
     NetworkRequest timeout(int ms) &&;
     NetworkRequest concurrent() &&;
     NetworkRequest authorizeTwitchV5(const QString &clientID,

src/singletons/Settings.hpp

@@ -273,6 +273,17 @@ public:
     // Custom URI Scheme
     QStringSetting customURIScheme = {"/external/urischeme"};
 
+    // Image Uploader
+    QStringSetting imageUploaderUrl = {"/external/imageUploader/url",
+                                       "https://i.nuuls.com/upload"};
+    QStringSetting imageUploaderFormField = {
+        "/external/imageUploader/formField", "attachment"};
+    QStringSetting imageUploaderHeaders = {"/external/imageUploader/headers",
+                                           ""};
+    QStringSetting imageUploaderLink = {"/external/imageUploader/link", ""};
+    QStringSetting imageUploaderDeletionLink = {
+        "/external/imageUploader/deletionLink", ""};
+
     /// Misc
     BoolSetting betaUpdates = {"/misc/beta", false};
 #ifdef Q_OS_LINUX

src/util/NuulsUploader.cpp

@@ -5,11 +5,15 @@
 #include "providers/twitch/TwitchMessageBuilder.hpp"
 #include "singletons/Paths.hpp"
 #include "singletons/Settings.hpp"
+#include "util/CombinePath.hpp"
 
 #include <QBuffer>
 #include <QHttpMultiPart>
+#include <QJsonArray>
+#include <QJsonDocument>
 #include <QMimeDatabase>
 #include <QMutex>
+#include <QSaveFile>
 
 #define UPLOAD_DELAY 2000
 // Delay between uploads in milliseconds
@@ -38,37 +42,76 @@ namespace chatterino {
 static auto uploadMutex = QMutex();
 static std::queue<RawImageData> uploadQueue;
 
-//logging information on successful uploads to a csv file
+// logging information on successful uploads to a json file
-void logToCsv(const QString originalFilePath, const QString link,
+void logToFile(const QString originalFilePath, QString imageLink,
-              ChannelPtr channel)
+               QString deletionLink, ChannelPtr channel)
 {
-    const QString csvFileName = (getSettings()->logPath.getValue().isEmpty()
+    const QString logFileName =
-                                     ? getPaths()->messageLogDirectory
+        combinePath((getSettings()->logPath.getValue().isEmpty()
-                                     : getSettings()->logPath) +
+                         ? getPaths()->messageLogDirectory
-                                "/ImageUploader.csv";
+                         : getSettings()->logPath),
-    QFile csvFile(csvFileName);
+                    "ImageUploader.json");
-    bool csvExisted = csvFile.exists();
+
-    bool isCsvOkay = csvFile.open(QIODevice::Append | QIODevice::Text);
+    //reading existing logs
-    if (!isCsvOkay)
+    QFile logReadFile(logFileName);
+    bool isLogFileOkay =
+        logReadFile.open(QIODevice::ReadWrite | QIODevice::Text);
+    if (!isLogFileOkay)
     {
         channel->addMessage(makeSystemMessage(
-            QString("Failed to open csv file with links at ") + csvFileName));
+            QString("Failed to open log file with links at ") + logFileName));
         return;
     }
-    QTextStream out(&csvFile);
+    auto logs = logReadFile.readAll();
-    qDebug() << csvExisted;
+    if (logs.isEmpty())
-    if (!csvExisted)
     {
-        out << "localPath,imageLink,timestamp,channelName\n";
+        logs = QJsonDocument(QJsonArray()).toJson();
     }
-    out << originalFilePath + QString(",") << link + QString(",")
+    logReadFile.close();
-        << QDateTime::currentSecsSinceEpoch()
+
-        << QString(",%1\n").arg(channel->getName());
+    //writing new data to logs
-    // image path (can be empty)
+    QJsonObject newLogEntry;
-    // image link
+    newLogEntry["channelName"] = channel->getName();
-    // timestamp
+    newLogEntry["deletionLink"] =
+        deletionLink.isEmpty() ? QJsonValue(QJsonValue::Null) : deletionLink;
+    newLogEntry["imageLink"] = imageLink;
+    newLogEntry["localPath"] = originalFilePath.isEmpty()
+                                   ? QJsonValue(QJsonValue::Null)
+                                   : originalFilePath;
+    newLogEntry["timestamp"] = QDateTime::currentSecsSinceEpoch();
     // channel name
-    csvFile.close();
+    // deletion link (can be empty)
+    // image link
+    // local path to an image (can be empty)
+    // timestamp
+    QSaveFile logSaveFile(logFileName);
+    logSaveFile.open(QIODevice::WriteOnly | QIODevice::Text);
+    QJsonArray entries = QJsonDocument::fromJson(logs).array();
+    entries.push_back(newLogEntry);
+    logSaveFile.write(QJsonDocument(entries).toJson());
+    logSaveFile.commit();
+}
+
+// extracting link to either image or its deletion from response body
+QString getJSONValue(QJsonValue responseJson, QString jsonPattern)
+{
+    for (const QString &key : jsonPattern.split("."))
+    {
+        responseJson = responseJson[key];
+    }
+    return responseJson.toString();
+}
+
+QString getLinkFromResponse(NetworkResult response, QString pattern)
+{
+    QRegExp regExp("\\{(.+)\\}");
+    regExp.setMinimal(true);
+    while (regExp.indexIn(pattern) != -1)
+    {
+        pattern.replace(regExp.cap(0),
+                        getJSONValue(response.parseJson(), regExp.cap(1)));
+    }
+    return pattern;
 }
 
 void uploadImageToNuuls(RawImageData imageData, ChannelPtr channel,
@@ -77,8 +120,15 @@ void uploadImageToNuuls(RawImageData imageData, ChannelPtr channel,
     const static char *const boundary = "thisistheboudaryasd";
     const static QString contentType =
         QString("multipart/form-data; boundary=%1").arg(boundary);
-    static QUrl url(Env::get().imageUploaderUrl);
+    QUrl url(getSettings()->imageUploaderUrl.getValue().isEmpty()
-    static QString formBody(Env::get().imageUploaderFormBody);
+                 ? getSettings()->imageUploaderUrl.getDefaultValue()
+                 : getSettings()->imageUploaderUrl);
+    QString formField(
+        getSettings()->imageUploaderFormField.getValue().isEmpty()
+            ? getSettings()->imageUploaderFormField.getDefaultValue()
+            : getSettings()->imageUploaderFormField);
+    QStringList extraHeaders(
+        getSettings()->imageUploaderHeaders.getValue().split(";"));
     QString originalFilePath = imageData.filePath;
 
     QHttpMultiPart *payload = new QHttpMultiPart(QHttpMultiPart::FormDataType);
@@ -90,32 +140,50 @@ void uploadImageToNuuls(RawImageData imageData, ChannelPtr channel,
                    QVariant(imageData.data.length()));
     part.setHeader(QNetworkRequest::ContentDispositionHeader,
                    QString("form-data; name=\"%1\"; filename=\"control_v.%2\"")
-                       .arg(formBody)
+                       .arg(formField)
                        .arg(imageData.format));
     payload->setBoundary(boundary);
     payload->append(part);
+
     NetworkRequest(url, NetworkRequestType::Post)
         .header("Content-Type", contentType)
-
+        .headerList(extraHeaders)
         .multiPart(payload)
         .onSuccess([&textEdit, channel,
                     originalFilePath](NetworkResult result) -> Outcome {
-            textEdit.insertPlainText(result.getData() + QString(" "));
+            QString link = getSettings()->imageUploaderLink.getValue().isEmpty()
+                               ? result.getData()
+                               : getLinkFromResponse(
+                                     result, getSettings()->imageUploaderLink);
+            QString deletionLink =
+                getSettings()->imageUploaderDeletionLink.getValue().isEmpty()
+                    ? ""
+                    : getLinkFromResponse(
+                          result, getSettings()->imageUploaderDeletionLink);
+            qDebug() << link << deletionLink;
+            textEdit.insertPlainText(link + " ");
             if (uploadQueue.empty())
             {
                 channel->addMessage(makeSystemMessage(
-                    QString("Your image has been uploaded to ") +
+                    QString("Your image has been uploaded to %1 %2.")
-                    result.getData()));
+                        .arg(link)
+                        .arg(deletionLink.isEmpty()
+                                 ? ""
+                                 : QString("(Deletion link: %1 )")
+                                       .arg(deletionLink))));
                 uploadMutex.unlock();
             }
             else
             {
                 channel->addMessage(makeSystemMessage(
-                    QString(
+                    QString("Your image has been uploaded to %1 %2. %3 left. "
-                        "Your image has been uploaded to %1 . %2 left. Please "
+                            "Please wait until all of them are uploaded. "
-                        "wait until all of them are uploaded. About %3 "
+                            "About %4 seconds left.")
-                        "seconds left.")
+                        .arg(link)
-                        .arg(result.getData() + QString(""))
+                        .arg(deletionLink.isEmpty()
+                                 ? ""
+                                 : QString("(Deletion link: %1 )")
+                                       .arg(deletionLink))
                         .arg(uploadQueue.size())
                         .arg(uploadQueue.size() * (UPLOAD_DELAY / 1000 + 1))));
                 // 2 seconds for the timer that's there not to spam the remote server
@@ -127,7 +195,7 @@ void uploadImageToNuuls(RawImageData imageData, ChannelPtr channel,
                 });
             }
 
-            logToCsv(originalFilePath, result.getData(), channel);
+            logToFile(originalFilePath, link, deletionLink, channel);
 
             return Success;
         })
@@ -161,7 +229,6 @@ void upload(const QMimeData *source, ChannelPtr channel,
         {
             QString localPath = path.toLocalFile();
             QMimeType mime = mimeDb.mimeTypeForUrl(path);
-            qDebug() << mime.name();
             if (mime.name().startsWith("image") && !mime.inherits("image/gif"))
             {
                 channel->addMessage(makeSystemMessage(

src/widgets/settingspages/ExternalToolsPage.cpp

@@ -3,8 +3,11 @@
 #include "Application.hpp"
 #include "util/Helpers.hpp"
 #include "util/LayoutCreator.hpp"
+#include "util/RemoveScrollAreaBackground.hpp"
 
+#include <QFormLayout>
 #include <QGroupBox>
+#include <QLabel>
 
 #define STREAMLINK_QUALITY \
     "Choose", "Source", "High", "Medium", "Low", "Audio only"
@@ -14,7 +17,12 @@ namespace chatterino {
 ExternalToolsPage::ExternalToolsPage()
 {
     LayoutCreator<ExternalToolsPage> layoutCreator(this);
-    auto layout = layoutCreator.setLayoutType<QVBoxLayout>();
+
+    auto scroll = layoutCreator.emplace<QScrollArea>();
+    auto widget = scroll.emplaceScrollAreaWidget();
+    removeScrollAreaBackground(scroll.getElement(), widget.getElement());
+
+    auto layout = widget.setLayoutType<QVBoxLayout>();
 
     {
         auto group = layout.emplace<QGroupBox>("Streamlink");
@@ -37,7 +45,7 @@ ExternalToolsPage::ExternalToolsPage()
         links->setTextFormat(Qt::RichText);
         links->setTextInteractionFlags(Qt::TextBrowserInteraction |
                                        Qt::LinksAccessibleByKeyboard |
-                                       Qt::LinksAccessibleByKeyboard);
+                                       Qt::LinksAccessibleByMouse);
         links->setOpenExternalLinks(true);
 
         groupLayout->setWidget(0, QFormLayout::SpanningRole, description);
@@ -85,6 +93,44 @@ ExternalToolsPage::ExternalToolsPage()
                                                getSettings()->customURIScheme));
     }
 
+    {
+        auto group = layout.emplace<QGroupBox>("Image Uploader");
+        auto groupLayout = group.setLayoutType<QFormLayout>();
+
+        const auto description = new QLabel(
+            "You can set custom host for uploading images, like "
+            "imgur.com or s-ul.eu.<br>Check " +
+            formatRichNamedLink("https://github.com/Chatterino/chatterino2/"
+                                "blob/master/docs/IMAGEUPLOADER.md",
+                                "this guide") +
+            " for help.");
+        description->setWordWrap(true);
+        description->setStyleSheet("color: #bbb");
+        description->setTextFormat(Qt::RichText);
+        description->setTextInteractionFlags(Qt::TextBrowserInteraction |
+                                             Qt::LinksAccessibleByKeyboard |
+                                             Qt::LinksAccessibleByMouse);
+        description->setOpenExternalLinks(true);
+
+        groupLayout->setWidget(0, QFormLayout::SpanningRole, description);
+
+        groupLayout->addRow(
+            "Request URL: ",
+            this->createLineEdit(getSettings()->imageUploaderUrl));
+        groupLayout->addRow(
+            "Form field: ",
+            this->createLineEdit(getSettings()->imageUploaderFormField));
+        groupLayout->addRow(
+            "Extra Headers: ",
+            this->createLineEdit(getSettings()->imageUploaderHeaders));
+        groupLayout->addRow(
+            "Image link: ",
+            this->createLineEdit(getSettings()->imageUploaderLink));
+        groupLayout->addRow(
+            "Deletion link: ",
+            this->createLineEdit(getSettings()->imageUploaderDeletionLink));
+    }
+
     layout->addStretch(1);
 }
 

src/widgets/splits/Split.cpp

@@ -207,33 +207,34 @@ Split::Split(QWidget *parent)
         [this] { this->focused.invoke(); });
     this->input_->ui_.textEdit->focusLost.connect(
         [this] { this->focusLost.invoke(); });
-    this->input_->ui_.textEdit->imagePasted.connect([this](const QMimeData
+    this->input_->ui_.textEdit->imagePasted.connect(
-                                                               *source) {
+        [this](const QMimeData *source) {
-        if (getSettings()->askOnImageUpload.getValue())
+            if (getSettings()->askOnImageUpload.getValue())
-        {
-            QMessageBox msgBox;
-            msgBox.setText("Image upload");
-            msgBox.setInformativeText(
-                "You are uploading an image to i.nuuls.com. You won't be able "
-                "to remove the image from the site. Are you okay with this?");
-            msgBox.addButton(QMessageBox::Cancel);
-            msgBox.addButton(QMessageBox::Yes);
-            msgBox.addButton("Yes, don't ask again", QMessageBox::YesRole);
-
-            msgBox.setDefaultButton(QMessageBox::Yes);
-
-            auto picked = msgBox.exec();
-            if (picked == QMessageBox::Cancel)
             {
-                return;
+                QMessageBox msgBox;
+                msgBox.setText("Image upload");
+                msgBox.setInformativeText(
+                    "You are uploading an image to an external server. You may "
+                    "not be able to remove the image from the site. Are you "
+                    "okay with this?");
+                msgBox.addButton(QMessageBox::Cancel);
+                msgBox.addButton(QMessageBox::Yes);
+                msgBox.addButton("Yes, don't ask again", QMessageBox::YesRole);
+
+                msgBox.setDefaultButton(QMessageBox::Yes);
+
+                auto picked = msgBox.exec();
+                if (picked == QMessageBox::Cancel)
+                {
+                    return;
+                }
+                else if (picked == 0)  // don't ask again button
+                {
+                    getSettings()->askOnImageUpload.setValue(false);
+                }
             }
-            else if (picked == 0)  // don't ask again button
+            upload(source, this->getChannel(), *this->input_->ui_.textEdit);
-            {
+        });
-                getSettings()->askOnImageUpload.setValue(false);
-            }
-        }
-        upload(source, this->getChannel(), *this->input_->ui_.textEdit);
-    });
     setAcceptDrops(true);
 }