Initial playlist-based custom soundtracks support (#9362)

* Initial playlist-based custom soundtracks support

This is the initial implementation of playlist-based (cellSearch) custom soundtracks support.
It is based on the initial work by @Megamouse and currently uses a directory-based approach to manage albums and utilizes FFMPEG to read audio metadata.

Background: The real PS3 can import music in XMB from USB into its internal storage (/dev_hdd0/music) and additionally stores metadata (artist, trackname, tracknumber, ...) in a database (/dev_hdd0/mms/db/metadata_db_hdd). Games can make use of imported music via cellSearch.

For the time being, this implementation does NOT make use of metadata_db_hdd as the db-format is not well understood and a folder-based approch is easier to use. Users only have to create folders inside /dev_hdd0/music and add music to it to create a "playlist". This playlists contents will be sorted alphabetically. As a result, users could prefix numbers to the audio-files to force a specific order.

The only really supported audio format is MP3. I also added support for AAC, AC3, WMA, ATRAC3 and ATRAC3 plus, however, non of these formats were successfully tested for several reasons. AC3 and WMA are not enabled in the current FFMPEG build which makes reading codec-specific data impossible. We could enable these later if we want to. AAC actually could work but I was not able to get it working in WipeOut HD Fury. My guess is that the game does not support AAC. Finally, I could not find any ATRAC3 (or Plus) music to test with.

This implementation currently only implements parts of cellSearchStartListSearch() and cellSearchStartContentSearchInList(). There are several other functions which are still completely unimplemented and will probably be needed by other games. However, this implementation is a starting-point and is enough for WipeOut and maybe a few other games.

A video which showcases this custom soundtrack support is available here: https://www.youtube.com/watch?v=4nu1OCtONTY

Next steps:
 - Utilize sortKey in cellSearchStartContentSearchInList()
 - Eliminate TODOs
 - Implement the missing other functions
 - Test on more games - I do not own many that support custom soundtracks

Signed-off-by: gladiac1337 <gladiac@gmail.com>

Co-authored-by: Megamouse <studienricky89@googlemail.com>
Co-authored-by: Ani <ani-leo@outlook.com>
Co-authored-by: Ivan <nekotekina@gmail.com>
This commit is contained in:
Chris 2020-12-31 20:47:09 +01:00 committed by GitHub
parent f1c61067bc
commit f8589de476
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

View File

@ -4,10 +4,22 @@
#include "Emu/Cell/PPUModule.h"
#include "cellMusic.h"
#include "cellSysutil.h"
#include <string>
#include "cellSearch.h"
#include "Utilities/StrUtil.h"
#ifdef _MSC_VER
#pragma warning(push, 0)
#else
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wold-style-cast"
#endif
extern "C" {
#include "libavformat/avformat.h"
#include "libavutil/dict.h"
}
LOG_CHANNEL(cellSearch);
template<>
@ -182,17 +194,24 @@ error_code cellSearchStartListSearch(CellSearchListSearchType type, CellSearchSo
return CELL_SEARCH_ERROR_PARAM;
}
const char* media_dir;
switch (type)
{
case CELL_SEARCH_LISTSEARCHTYPE_MUSIC_ALBUM:
case CELL_SEARCH_LISTSEARCHTYPE_MUSIC_GENRE:
case CELL_SEARCH_LISTSEARCHTYPE_MUSIC_ARTIST:
case CELL_SEARCH_LISTSEARCHTYPE_MUSIC_PLAYLIST:
media_dir = "music";
break;
case CELL_SEARCH_LISTSEARCHTYPE_PHOTO_YEAR:
case CELL_SEARCH_LISTSEARCHTYPE_PHOTO_MONTH:
case CELL_SEARCH_LISTSEARCHTYPE_PHOTO_ALBUM:
case CELL_SEARCH_LISTSEARCHTYPE_PHOTO_PLAYLIST:
media_dir = "photo";
break;
case CELL_SEARCH_LISTSEARCHTYPE_VIDEO_ALBUM:
case CELL_SEARCH_LISTSEARCHTYPE_MUSIC_PLAYLIST:
media_dir = "video";
break;
default:
return CELL_SEARCH_ERROR_PARAM;
@ -223,11 +242,173 @@ error_code cellSearchStartListSearch(CellSearchListSearchType type, CellSearchSo
const u32 id = *outSearchId = idm::make<search_object_t>();
sysutil_register_cb([=](ppu_thread& ppu) -> s32
sysutil_register_cb([=, content_map = g_fxo->get<ContentIdMap>()](ppu_thread& ppu) -> s32
{
auto curr_search = idm::get<search_object_t>(id);
vm::var<CellSearchResultParam> resultParam;
resultParam->searchId = id;
resultParam->resultNum = 0; // TODO
resultParam->resultNum = 0; // Set again later
std::function<void(const std::string&)> searchInFolder = [&, type](const std::string& vpath)
{
// TODO: this is just a workaround. On a real PS3 the playlists seem to be stored in dev_hdd0/mms/db/metadata_db_hdd
std::vector<fs::dir_entry> dirs_sorted;
for (auto&& entry : fs::dir(vfs::get(vpath)))
{
entry.name = vfs::unescape(entry.name);
if (entry.is_directory)
{
if (entry.name == "." || entry.name == "..")
{
continue; // these dirs are not included in the dir list
}
dirs_sorted.push_back(entry);
}
}
// clang-format off
std::sort(dirs_sorted.begin(), dirs_sorted.end(), [&](const fs::dir_entry& a, const fs::dir_entry& b) -> bool
{
switch (sortOrder)
{
case CELL_SEARCH_SORTORDER_ASCENDING:
// Order alphabetically ascending
return a.name < b.name;
case CELL_SEARCH_SORTORDER_DESCENDING:
// Order alphabetically descending
return a.name > b.name;
default:
{
return false;
}
}
});
// clang-format on
for (auto&& item : dirs_sorted)
{
item.name = vfs::unescape(item.name);
if (item.name == "." || item.name == "..")
{
continue;
}
if (!item.is_directory)
{
continue;
}
const std::string item_path(vpath + "/" + item.name);
// Count files
u32 numOfItems = 0;
for (auto&& file : fs::dir(vfs::get(item_path)))
{
file.name = vfs::unescape(file.name);
if (file.name == "." || file.name == ".." || file.is_directory)
{
continue;
}
numOfItems++;
}
const u64 hash = std::hash<std::string>()(item_path);
auto found = content_map->find(hash);
if (found == content_map->end()) // content isn't yet being tracked
{
auto ext_offset = item.name.find_last_of('.'); // used later if no "Title" found
std::shared_ptr<search_content_t> curr_find = std::make_shared<search_content_t>();
if (item_path.length() > CELL_SEARCH_PATH_LEN_MAX)
{
// TODO: Create mapping which will be resolved to an actual hard link in VFS by cellSearchPrepareFile
cellSearch.warning("cellSearchStartListSearch(): Directory-Path \"%s\" is too long and will be omitted: %i", item_path, item_path.length());
continue;
// std::string link = "/.tmp/" + std::to_string(hash) + item.name.substr(ext_offset);
// strcpy_trunc(curr_find->infoPath.contentPath, link);
// std::lock_guard lock(search->links_mutex);
// search->content_links.emplace(std::move(link), item_path);
}
else
{
strcpy_trunc(curr_find->infoPath.contentPath, item_path);
}
if (item.name.size() > CELL_SEARCH_TITLE_LEN_MAX)
{
item.name.resize(CELL_SEARCH_TITLE_LEN_MAX);
}
switch (type)
{
case CELL_SEARCH_LISTSEARCHTYPE_MUSIC_ALBUM:
case CELL_SEARCH_LISTSEARCHTYPE_MUSIC_GENRE:
case CELL_SEARCH_LISTSEARCHTYPE_MUSIC_ARTIST:
case CELL_SEARCH_LISTSEARCHTYPE_MUSIC_PLAYLIST:
{
curr_find->type = CELL_SEARCH_CONTENTTYPE_MUSICLIST;
CellSearchMusicListInfo& info = curr_find->data.music_list;
info.listType = type; // CellSearchListType matches CellSearchListSearchType
info.numOfItems = numOfItems;
info.duration = 0;
strcpy_trunc(info.title, item.name);
strcpy_trunc(info.artistName, "ARTIST NAME");
break;
}
case CELL_SEARCH_LISTSEARCHTYPE_PHOTO_YEAR:
case CELL_SEARCH_LISTSEARCHTYPE_PHOTO_MONTH:
case CELL_SEARCH_LISTSEARCHTYPE_PHOTO_ALBUM:
case CELL_SEARCH_LISTSEARCHTYPE_PHOTO_PLAYLIST:
{
curr_find->type = CELL_SEARCH_CONTENTTYPE_PHOTOLIST;
CellSearchPhotoListInfo& info = curr_find->data.photo_list;
info.listType = type; // CellSearchListType matches CellSearchListSearchType
info.numOfItems = numOfItems;
strcpy_trunc(info.title, item.name);
break;
}
case CELL_SEARCH_LISTSEARCHTYPE_VIDEO_ALBUM:
{
curr_find->type = CELL_SEARCH_CONTENTTYPE_VIDEOLIST;
CellSearchVideoListInfo& info = curr_find->data.video_list;
info.listType = type; // CellSearchListType matches CellSearchListSearchType
info.numOfItems = numOfItems;
info.duration = 0;
strcpy_trunc(info.title, item.name);
break;
}
default:
{
break;
}
}
content_map->emplace(hash, curr_find);
curr_search->content_ids.emplace_back(hash, curr_find); // place this file's "ID" into the list of found types
cellSearch.notice("cellSearchStartListSearch(): Content ID: %08X Path: \"%s\"", hash, item_path);
}
else // list is already stored and tracked
{
// TODO
// Perform checks to see if the identified list has been modified since last checked
// In which case, update the stored content's properties
// auto content_found = &content_map->at(content_id);
// curr_search->content_ids.emplace_back(found->first, found->second);
}
}
};
searchInFolder(fmt::format("/dev_hdd0/%s", media_dir));
resultParam->resultNum = ::narrow<s32>(curr_search->content_ids.size());
search->state.store(search_state::idle);
search->func(ppu, CELL_SEARCH_EVENT_LISTSEARCH_RESULT, CELL_OK, vm::cast(resultParam.addr()), search->userData);
@ -286,13 +467,340 @@ error_code cellSearchStartContentSearchInList(vm::cptr<CellSearchContentId> list
return CELL_SEARCH_ERROR_GENERIC;
}
const auto content_map = g_fxo->get<ContentIdMap>();
auto found = content_map->find(*reinterpret_cast<const u64*>(listId->data));
if (found == content_map->end())
{
// content ID not found, perform a search first
return CELL_SEARCH_ERROR_CONTENT_NOT_FOUND;
}
CellSearchContentSearchType type = CELL_SEARCH_CONTENTSEARCHTYPE_NONE;
const auto& content_info = found->second;
switch (content_info->type)
{
case CELL_SEARCH_CONTENTTYPE_MUSICLIST:
type = CELL_SEARCH_CONTENTSEARCHTYPE_MUSIC_ALL;
break;
case CELL_SEARCH_CONTENTTYPE_PHOTOLIST:
type = CELL_SEARCH_CONTENTSEARCHTYPE_PHOTO_ALL;
break;
case CELL_SEARCH_CONTENTTYPE_VIDEOLIST:
type = CELL_SEARCH_CONTENTSEARCHTYPE_VIDEO_ALL;
break;
case CELL_SEARCH_CONTENTTYPE_MUSIC:
case CELL_SEARCH_CONTENTTYPE_PHOTO:
case CELL_SEARCH_CONTENTTYPE_VIDEO:
case CELL_SEARCH_CONTENTTYPE_SCENE:
default:
return CELL_SEARCH_ERROR_NOT_LIST;
}
const u32 id = *outSearchId = idm::make<search_object_t>();
sysutil_register_cb([=](ppu_thread& ppu) -> s32
sysutil_register_cb([=, list_path = std::string(content_info->infoPath.contentPath)](ppu_thread& ppu) -> s32
{
auto curr_search = idm::get<search_object_t>(id);
vm::var<CellSearchResultParam> resultParam;
resultParam->searchId = id;
resultParam->resultNum = 0; // TODO
resultParam->resultNum = 0; // Set again later
std::function<void(const std::string&)> searchInFolder = [&, type](const std::string& vpath)
{
std::vector<fs::dir_entry> files_sorted;
for (auto&& entry : fs::dir(vfs::get(vpath)))
{
entry.name = vfs::unescape(entry.name);
if (entry.is_directory || entry.name == "." || entry.name == "..")
{
continue;
}
files_sorted.push_back(entry);
}
// clang-format off
std::sort(files_sorted.begin(), files_sorted.end(), [&](const fs::dir_entry& a, const fs::dir_entry& b) -> bool
{
switch (sortOrder)
{
case CELL_SEARCH_SORTORDER_ASCENDING:
// Order alphabetically ascending
return a.name < b.name;
case CELL_SEARCH_SORTORDER_DESCENDING:
// Order alphabetically descending
return a.name > b.name;
default:
{
return false;
}
}
});
// clang-format on
// TODO: Use sortKey (CellSearchSortKey) to allow for sorting by category
for (auto&& item : files_sorted)
{
// TODO
// Perform first check that file is of desired type. For example, don't wanna go
// identifying "AlbumArt.jpg" as an MP3. Hrm... Postpone this thought. Do games
// perform their own checks? DIVA ignores anything without the MP3 extension.
const std::string item_path(vpath + "/" + item.name);
const u64 hash = std::hash<std::string>()(item_path);
auto found = content_map->find(hash);
if (found == content_map->end()) // content isn't yet being tracked
{
auto ext_offset = item.name.find_last_of('.'); // used later if no "Title" found
std::shared_ptr<search_content_t> curr_find = std::make_shared<search_content_t>();
if (item_path.length() > CELL_SEARCH_PATH_LEN_MAX)
{
// Create mapping which will be resolved to an actual hard link in VFS by cellSearchPrepareFile
std::string link = "/.tmp/" + std::to_string(hash) + item.name.substr(ext_offset);
strcpy_trunc(curr_find->infoPath.contentPath, link);
std::lock_guard lock(search->links_mutex);
search->content_links.emplace(std::move(link), item_path);
}
else
{
strcpy_trunc(curr_find->infoPath.contentPath, item_path);
}
// TODO - curr_find.infoPath.thumbnailPath
if (type == CELL_SEARCH_CONTENTSEARCHTYPE_MUSIC_ALL)
{
curr_find->type = CELL_SEARCH_CONTENTTYPE_MUSIC;
CellSearchMusicInfo& info = curr_find->data.music;
// Only print FFMPEG errors, fatals and panics
av_log_set_level(AV_LOG_ERROR);
AVDictionary* av_dict_opts = nullptr;
av_dict_set(&av_dict_opts, "probesize", "96", 0);
AVFormatContext* av_format_ctx = nullptr;
av_format_ctx = avformat_alloc_context();
// Open input file
if (avformat_open_input(&av_format_ctx, (vfs::get(vpath) + "/" + item.name).c_str(), 0, &av_dict_opts) < 0)
{
// Failed to open file
av_dict_free(&av_dict_opts);
avformat_free_context(av_format_ctx);
continue;
}
av_dict_free(&av_dict_opts);
// Find stream information
if (avformat_find_stream_info(av_format_ctx, 0) < 0)
{
// Failed to load stream information
avformat_free_context(av_format_ctx);
continue;
}
// Derive first audio stream id from avformat context
int stream_index = -1;
for (uint i = 0; i < av_format_ctx->nb_streams; i++)
{
if (av_format_ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_AUDIO)
{
stream_index = i;
break;
}
}
if (stream_index == -1)
{
// Failed to find an audio stream
avformat_free_context(av_format_ctx);
continue;
}
AVStream* stream = av_format_ctx->streams[stream_index];
AVCodecParameters* codec = stream->codecpar;
info.bitrate = codec->bit_rate / 1000; // TODO: Assumption, verify value
info.quantizationBitrate = codec->bit_rate / 1000; // TODO: Assumption, verify value
info.samplingRate = codec->sample_rate; // TODO: Assumption, verify value
info.drmEncrypted = 0; // Needs to be 0 or it wont be accepted
info.duration = av_format_ctx->duration / 1000; // TODO: Assumption, verify value
info.releasedYear = 0; // TODO: Use "date" id3 tag for this
info.size = item.size;
info.playCount = 0; // we do not track this for now
info.lastPlayedDate = 0; // we do not track this for now
info.importedDate = 0; // we do not track this for now
info.status = CELL_SEARCH_CONTENTSTATUS_AVAILABLE; // CellSearchContentStatus
switch (codec->codec_id) // AVCodecID
{
case AV_CODEC_ID_MP3:
info.codec = CELL_SEARCH_CODEC_MP3; // CellSearchCodec
break;
case AV_CODEC_ID_AAC:
info.codec = CELL_SEARCH_CODEC_AAC; // CellSearchCodec
break;
case AV_CODEC_ID_AC3:
info.codec = CELL_SEARCH_CODEC_AC3; // CellSearchCodec
break;
case AV_CODEC_ID_WMAV1:
case AV_CODEC_ID_WMAV2:
info.codec = CELL_SEARCH_CODEC_WMA; // CellSearchCodec
break;
case AV_CODEC_ID_ATRAC3:
info.codec = CELL_SEARCH_CODEC_AT3; // CellSearchCodec
break;
case AV_CODEC_ID_ATRAC3P:
info.codec = CELL_SEARCH_CODEC_AT3P; // CellSearchCodec
break;
default:
info.codec = CELL_SEARCH_CODEC_UNKNOWN; // CellSearchCodec
info.status = CELL_SEARCH_CONTENTSTATUS_NOT_SUPPORTED; // CellSearchContentStatus
break;
}
AVDictionaryEntry *tag;
std::string value;
info.trackNumber = 0;
tag = av_dict_get(av_format_ctx->metadata, "track", 0, AV_DICT_IGNORE_SUFFIX);
if (tag != nullptr)
{
std::string tmp(tag->value);
info.trackNumber = stoi(tmp.substr(0, tmp.find("/")));
}
tag = av_dict_get(av_format_ctx->metadata, "album", 0, AV_DICT_IGNORE_SUFFIX);
if (tag != nullptr)
{
value = tag->value;
if (value.size() > CELL_SEARCH_TITLE_LEN_MAX)
{
value.resize(CELL_SEARCH_TITLE_LEN_MAX);
}
strcpy_trunc(info.albumTitle, value);
}
else
{
strcpy_trunc(info.albumTitle, "Unknown Album");
}
tag = av_dict_get(av_format_ctx->metadata, "title", 0, AV_DICT_IGNORE_SUFFIX);
if (tag != nullptr)
{
value = tag->value;
if (value.size() > CELL_SEARCH_TITLE_LEN_MAX)
{
value.resize(CELL_SEARCH_TITLE_LEN_MAX);
}
strcpy_trunc(info.title, value);
}
else
{
// Fall back to filename
value = item.name.substr(0, ext_offset);
if (value.size() > CELL_SEARCH_TITLE_LEN_MAX)
{
value.resize(CELL_SEARCH_TITLE_LEN_MAX);
strcpy_trunc(info.title, value);
}
else
{
strcpy_trunc(info.title, value);
}
}
tag = av_dict_get(av_format_ctx->metadata, "artist", 0, AV_DICT_IGNORE_SUFFIX);
if (tag != nullptr)
{
value = tag->value;
if (value.size() > CELL_SEARCH_TITLE_LEN_MAX)
{
value.resize(CELL_SEARCH_TITLE_LEN_MAX);
}
strcpy_trunc(info.artistName, value);
}
else
{
strcpy_trunc(info.artistName, "Unknown Artist");
}
tag = av_dict_get(av_format_ctx->metadata, "genre", 0, AV_DICT_IGNORE_SUFFIX);
if (tag != nullptr)
{
value = tag->value;
if (value.size() > CELL_SEARCH_TITLE_LEN_MAX)
{
value.resize(CELL_SEARCH_TITLE_LEN_MAX);
}
strcpy_trunc(info.genreName, value);
}
else
{
strcpy_trunc(info.genreName, "Unknown Genre");
}
avformat_close_input(&av_format_ctx);
avformat_free_context(av_format_ctx);
}
else if (type == CELL_SEARCH_CONTENTSEARCHTYPE_PHOTO_ALL)
{
curr_find->type = CELL_SEARCH_CONTENTTYPE_PHOTO;
CellSearchPhotoInfo& info = curr_find->data.photo;
// TODO - Some kinda file photo analysis and assign the values as such
info.size = item.size;
info.importedDate = 0;
info.takenDate = 0;
info.width = 0;
info.height = 0;
info.orientation = 0; // CellSearchOrientation
info.codec = 0; // CellSearchCodec
info.status = 0; // CellSearchContentStatus
strcpy_trunc(info.title, item.name.substr(0, ext_offset));
strcpy_trunc(info.albumTitle, "ALBUM TITLE");
}
else if (type == CELL_SEARCH_CONTENTSEARCHTYPE_VIDEO_ALL)
{
curr_find->type = CELL_SEARCH_CONTENTTYPE_VIDEO;
CellSearchVideoInfo& info = curr_find->data.video;
// TODO - Some kinda file video analysis and assign the values as such
info.duration = 0;
info.size = item.size;
info.importedDate = 0;
info.takenDate = 0;
info.videoBitrate = 0;
info.audioBitrate = 0;
info.playCount = 0;
info.drmEncrypted = 0;
info.videoCodec = 0; // CellSearchCodec
info.audioCodec = 0; // CellSearchCodec
info.status = 0; // CellSearchContentStatus
strcpy_trunc(info.title, item.name.substr(0, ext_offset)); // it'll do for the moment...
strcpy_trunc(info.albumTitle, "ALBUM TITLE");
}
content_map->emplace(hash, curr_find);
curr_search->content_ids.emplace_back(hash, curr_find); // place this file's "ID" into the list of found types
cellSearch.notice("cellSearchStartContentSearchInList(): Content ID: %08X Path: \"%s\"", hash, item_path);
}
else // file is already stored and tracked
{
// TODO
// Perform checks to see if the identified file has been modified since last checked
// In which case, update the stored content's properties
// auto content_found = &content_map->at(content_id);
curr_search->content_ids.emplace_back(found->first, found->second);
}
}
};
searchInFolder(list_path);
resultParam->resultNum = ::narrow<s32>(curr_search->content_ids.size());
search->state.store(search_state::idle);
search->func(ppu, CELL_SEARCH_EVENT_CONTENTSEARCH_INLIST_RESULT, CELL_OK, vm::cast(resultParam.addr()), search->userData);